We prove that all moment varieties of univariate Gaussian mixtures have the expected dimension. Our approach rests on intersection theory and Terracini's classification of defective surfaces. The analogous identifiability result is shown to be false for mixtures of Gaussians in dimension three and higher. Their moments up to third order define projective varieties that are defective. Our geometric study suggests an extension of the Alexander-Hirschowitz Theorem for Veronese varieties to the Gaussian setting.