We prove a conjecture of Fyodorov and Keating on the supercritical moments of the partition function of the C$β$E field or equivalently the supercritical moments of moments of the characteristic polynomial of the C$β$E ensemble for general $β>0$ and general real moment exponents. Moreover, we give the first expression for all correlation functions of the $\mathsf{Sine}_β$ point process for all $β>0$. The main object behind both results is the Hua-Pickrell stochastic zeta function introduced by Li and Valkó.