We study generalized Hermite polynomials with rectangular matrix arguments arising in multivariate statistical analysis and the theory of zonal polynomials. We show that these are well-suited for expressing the Wiener-Ito chaos expansion of functionals of the spectral measure associated with Gaussian matrices. In particular, we obtain the Wiener chaos expansion of Gaussian determinants of the form $\det(XX^T)^{1/2}$ and prove that, in the setting where the rows of $X$ are i.i.d. centred Gaussian vectors with a given covariance matrix, its projection coefficients admit a geometric interpretation in terms of intrinsic volumes of ellipsoids, thus extending the content of Kabluchko and Zaporozhets (2012) to arbitrary chaotic projection coefficients. Our proofs are based on a crucial relation between generalized Hermite polynomials and generalized Laguerre polynomials. In a second part, we introduce the matrix analog of the classical Mehler's formula for the Ornstein-Uhlenbeck semigroup and prove that matrix-variate Hermite polynomials are eigenfunctions of these operators. As a byproduct, we derive an orthogonality relation for Hermite polynomials evaluated at correlated Gaussian matrices. We apply our results to vectors of independent arithmetic random waves on the three-torus, proving in particular a CLT in the high-energy regime for a generalized notion of total variation on the full torus.