We relate the combinatorics of Hall-Littlewood polynomials to that of abelian $p$-groups with alternating or Hermitian perfect pairings. Our main result is an analogue of the classical relationship between the Hall algebra of abelian $p$-groups (without pairings) and Hall-Littlewood polynomials. Specifically, we define a module over the classical Hall algebra with basis indexed by groups with pairings, and explicitly relate its structure constants to Hall-Littlewood polynomials at different values of the parameter $t$. We also show certain expectation formulas with respect to Cohen-Lenstra type measures on groups with pairings. In the alternating case this gives a new and simpler proof of previous results of Delaunay-Jouhet.