We identify the fluctuations of the partition function for a class of random energy models, where the energies are given by the positions of the particles of the complex-valued branching Brownian motion (BBM). Specifically, we provide the weak limit theorems for the partition function in the so-called "glassy phase" -- the regime of parameters, where the behaviour of the partition function is governed by the extrema of BBM. We allow for arbitrary correlations between the real and imaginary parts of the energies. This extends the recent result of Madaule, Rhodes and Vargas, where the uncorrelated case was treated. In particular, our result covers the case of the real-valued BBM energy model at complex temperatures.