We first consider nested CLE$_4$ in a simply-connected domain and compute some exact renormalised probabilities: the probability that two points belong to the same CLE$_4$ gasket and the probability that two points belong to the outermost CLE$_4$ gasket. The resulting conformally covariant formulas have a non-trivial modular dependence, expressed explicitly in terms of Jacobi theta functions. We are guided by the conformal field theory of the Ashkin-Teller (AT) model, but our proofs are purely probabilistic using Brownian loop soups, gauge-twisted Gaussian free fields (GFFs) and the level line geometry of the 2D continuum GFF. More generally, we also calculate renormalised probabilities that two points belong to CLE$_4$ gaskets sampled in alternation with certain two-valued sets of the Gaussian free field. These quantities correspond to the two-point function of the conjectured scaling limit of the AT single spins on the critical line. At the decoupling point, our results recover the Ising model correlations; they also suggest a CLE$_4$-based continuum FK representation of the AT single-spin fields along a whole segment of the critical line.