We study the secular coefficients of $N \times N$ random unitary matrices $U_{N}$ drawn from the Circular $β$-Ensemble, which are defined as the coefficients of $\{z^n\}$ in the characteristic polynomial $\det(1-zU_{N}^{*})$. When $β> 4$ we obtain a new class of limiting distributions that arise when both $n$ and $N$ tend to infinity simultaneously. We solve an open problem of Diaconis and Gamburd by showing that for $β=2$, the middle coefficient tends to zero as $N \to \infty$. We show how the theory of Gaussian multiplicative chaos (GMC) plays a prominent role in these problems and in the explicit description of the obtained limiting distributions. We extend the remarkable magic square formula of Diaconis and Gamburd for the moments of secular coefficients to all $β>0$ and analyse the asymptotic behaviour of the moments. We obtain estimates on the order of magnitude of the secular coefficients for all $β> 0,$ and these estimates are sharp when $β\geq 2$. These insights motivated us to introduce a new stochastic object associated with the secular coefficients, which we call Holomorphic Multiplicative Chaos (HMC). Viewing the HMC as a random distribution, we prove a sharp result about its regularity in an appropriate Sobolev space. Our proofs expose and exploit several novel connections with other areas, including random permutations, Tauberian theorems and combinatorics.