In a recent breakthrough [arXiv:2301.04112], Chatterjee proved site disorder chaos in the Edwards-Anderson (EA) short-range spin glass model utilizing the Hermite spectral method. In this paper, we demonstrate the further usefulness of this Hermite spectral approach by extending the validity of site disorder chaos in three related spin glass models. The first, called the mixed even $p$-spin short-range model, is a generalization of the EA model where the underlying graph is a deterministic bounded degree hypergraph consisting of hyperedges with even number of vertices. The second model is the diluted mixed $p$-spin model, which is allowed to have hyperedges with both odd and even number of vertices. For both models, our results hold under general symmetric disorder distributions. The main novelty of our argument is played by an elementary algebraic equation for the Fourier-Hermite series coefficients for the two-spin correlation functions. It allows us to deduce necessary geometric conditions to determine the contributing coefficients in the overlap function, which in spirit is the same as the crucial Lemma 1 in [arXiv:2301.04112]. Finally, we also establish disorder chaos in the Lévy model with stable index $α\in (1, 2)$.