In this paper, we prove universality of the distribution of the cokernels of a random $p$-adic matrix with inhomogeneously balanced columns. More precisely, let $u \ge 0$ be an integer and $A(n)$ be a random $n \times (n+u)$ matrix over $\mathbb{Z}_p$ whose $i$-th column is $α_n(i)$-balanced. We prove that if $\sum_{i=1}^{n+u} \exp(-εα_n(i)n) \to 0$ as $n \to \infty$ for every $ε>0$, then the cokernels of $A(n)$ converge in distribution, as $n \to \infty$, to the same limiting law as the cokernels of Haar-random $n \times (n+u)$ matrices over $\mathbb{Z}_p$. This extends a universality theorem of Nguyen and Wood to random $p$-adic matrices with inhomogeneously balanced columns.