In this paper, we study the value sets of non-permutation polynomial functions over the residue class ring $\mathbb{Z}/m\mathbb{Z}$. When $m=p^r$ is a power of some prime $p$, an upper bound is given for the size of the value set of a polynomial function which is not a permutation. We also show that this upper bound can be achieved by some integral polynomials. Finally, we generalize the results for any positive integer $m$ with known prime decomposition.