We consider random lower triangular matrices such that the entries on and below the diagonal are i.i.d. copies of some $\mathbb{Z}$-valued random variable. We prove that the Sylow $p$-subgroups of the cokernels of these matrices have the same constant order fluctuations as that of the matrix products studied by Nguyen and Van Peski. As a special case, we can describe the limiting fluctuations of the rank of lower triangular matrices over $\mathbb{F}_p$ with i.i.d. random entries on and below the diagonal.