We define and analyze low-rank parity-check (LRPC) codes over extension rings of the finite chain ring $\mathbb{Z}_{p^r}$, where $p$ is a prime and $r$ is a positive integer. LRPC codes have originally been proposed by Gaborit et al.(2013) over finite fields for cryptographic applications. The adaption to finite rings is inspired by a recent paper by Kamche et al. (2019), which constructed Gabidulin codes over finite principle ideal rings with applications to space-time codes and network coding. We give a decoding algorithm based on simple linear-algebraic operations. Further, we derive an upper bound on the failure probability of the decoder. The upper bound is valid for errors whose rank is equal to the free rank.