We evaluate the high temperature limit of the free energy of spin glasses on the hypercube with Hamiltonian $H_N(σ) = σ^T J σ$, where the coupling matrix $J$ is drawn from certain symmetric orthogonally invariant ensembles. Our derivation relates the annealed free energy of these models to a spherical integral, and expresses the limit of the free energy in terms of the limiting spectral measure of the coupling matrix $J$. As an application, we derive the limiting free energy of the Random Orthogonal Model (ROM) at high temperatures, which confirms non-rigorous calculations of Marinari et al. (1994). Our methods also apply to other well-known models of disordered systems, including the SK and Gaussian Hopfield models.