We prove that the boundary of the Hall-Littlewood $t$-deformation of the Gelfand-Tsetlin graph is parametrized by infinite integer signatures, extending results of Gorin and Cuenca on boundaries of related deformed Gelfand-Tsetlin graphs. In the special case when $1/t$ is a prime $p$ we use this to recover results of Bufetov-Qiu and Assiotis on infinite $p$-adic random matrices, placing them in the general context of branching graphs derived from symmetric functions. Our methods rely on explicit formulas for certain skew Hall-Littlewood polynomials. As a separate corollary to these, we obtain a simple expression for the joint distribution of the cokernels of products $A_1, A_2A_1, A_3A_2A_1,\ldots$ of independent Haar-distributed matrices $A_i$ over the $p$-adic integers $\mathbb{Z}_p$. This expression generalizes the explicit formula for the classical Cohen-Lenstra measure on abelian $p$-groups.