Given a prime $p$, let $P(t)$ be a non-constant monic polynomial in $t$ over the ring $\mathbb{Z}_{p}$ of $p$-adic integers. Let $X_{n}$ be an $n \times n$ random matrix over $\mathbb{Z}_{p}$ with independent entries that lie in any residue class modulo $p$ with probability at most $1 - ε$ for a fixed real number $0 < ε< 1$. We prove that as $n \rightarrow \infty$, the distribution of the cokernel $\mathrm{cok}(P(X_{n}))$ of $P(X_{n})$ converges to the distribution given by a finite product of some explicit measures that resemble Cohen--Lenstra measures. For example, the random matrix $X_{n}$ can be taken as a Haar-random matrix or a uniformly random $(0,1)$-matrix. We consider the distribution of $\mathrm{cok}(P(X_{n}))$ as a distribution of modules over $\mathbb{Z}_{p}[t]/(P(t))$, which gives us a clearer formulation in comparison to considering the distribution as that of abelian groups. For the proof, we first reduce our problem into a problem over $\mathbb{Z}/p^{k}\mathbb{Z}$, for large enough positive integer $k$, in place of $\mathbb{Z}_{p}$. Then we use a result of Sawin and Wood to reduce our problem into another problem of computing the limit of the expected number of surjective $(\mathbb{Z}/p^{k}\mathbb{Z})[t]/(P(t))$-linear maps from $\mathrm{cok}(P(X_{n}))$ modulo $p^{k}$ to a fixed finite size $(\mathbb{Z}/p^{k}\mathbb{Z})[t]/(P(t))$-module $G$. To estimate the expected number and compute the desired limit, we carefully adopt subtle techniques developed by Wood, which were originally used to compute the asymptotic distribution of the $p$-part of the sandpile group of a random graph.