This paper establishes a CLT for linear statistics of the form $\langle \mathbf{q},\boldsymbolσ \rangle$ with quantitative Berry-Esseen bounds, where $\boldsymbolσ$ is an observation from an exponential family with a quadratic form as its sufficient statistic, in the \enquote{high-temperature} regime. We apply our general result to random field Ising models with both discrete and continuous spins. To demonstrate the generality of our techniques, we apply our results to derive both quenched and annealed CLTs in various examples, which include Ising models on some graph ensembles of common interest (Erdős-Rényi, regular, dense bipartite), and the Hopfield spin glass model. Our proofs rely on a combination of Stein's method of exchangeable pairs and Chevet type concentration inequalities.