We study an irreducible Markov chain on the category of finite abelian $p$-groups, whose stationary measure is the Cohen-Lenstra distribution. This Markov chain arises when one studies the cokernel of a random matrix $M$, after conditioning on a submatrix of $M$. We show two surprising facts about this Markov chain. Firstly, it is reversible. Hence, one may regard it is a random walk on finite abelian $p$-groups. The proof of reversibility also explains the appearance of the Cohen-Lenstra distribution in the context of random matrices. Secondly, we can explicitly determine the spectrum of the infinite transition matrix associated to this Markov chain.