An $m$-ovoid of a finite polar space $\mathcal{P}$ is a set $\mathcal{O}$ of points such that every maximal subspace of $\mathcal{P}$ contains exactly $m$ points of $\mathcal{O}$. In the case when $\mathcal{P}$ is an elliptic quadric $\mathcal{Q}^-(2r+1, q)$ of rank $r$ in $\mathbb{F}_q^{2r+2}$, we prove that an $m$-ovoid exists only if $m$ satisfies a certain modular equality, which depends on $q$ and $r$. This condition rules out many of the possible values of $m$. Previously, only a lower bound on $m$ was known, which we slightly improve as a byproduct of our method. We also obtain a characterization of the $m$-ovoids of $\mathcal{Q}^{-}(7,q)$ for $q = 2$ and $(m, q) = (4, 3)$.