We prove that given any $ε>0$, random integral $n\times n$ matrices with independent entries that lie in any residue class modulo a prime with probability at most $1-ε$ have cokernels asymptotically (as $n\rightarrow\infty$) distributed as in the distribution on finite abelian groups that Cohen and Lenstra conjecture as the distribution for class groups of imaginary quadratic fields. This is a refinement of a result on the distribution of ranks of random matrices with independent entries in $\mathbb{Z}/p\mathbb{Z}$. This is interesting especially in light of the fact that these class groups are naturally cokernels of square matrices. We also prove the analogue for $n\times (n+u)$ matrices.