The Hermite polynomials are ubiquitous but can be difficult to work with due to their unwieldy definition in terms of derivatives. To remedy this, we showcase an underappreciated Gaussian integral formula for the Hermite polynomials, which is especially useful for generalizing to multivariable Hermite polynomials. Taking this as our definition, we prove many useful consequences, including: 1. Combinatorial interpretations for the Hermite polynomials, including a proof of orthogonality. 2. A more elementary proof of Plancherel--Rotach asymptotics that does not involve residues. 3. Limit theorems for GUE random matrices and Dyson's Brownian motion, including bulk convergence to the semi-circle law and edge convergence to the Airy limit/Tracy-Widom law. 4. An analysis of a phase transition in the spiked GUE random matrix as the top eigenvalue goes from well-separated to attached to the bulk, analogous to the BBP phase transition. 5. Elementary derivations of Edgeworth expansions and multivariable Edgeworth expansions. This article is primarily expository and features many illustrative figures.