A cover by left ideals of an associative (not necessarily commutative or unital) ring $R$ is a collection of proper left ideals whose set-theoretic union equals $R$. If such a cover exists, then $η_\ell(R)$ is the cardinality of a minimal cover, and $R$ is $η_\ell$-elementary if $η_\ell(R)<η_\ell(R/I)$ for every nonzero two-sided ideal $I$ of $R$. We classify all $η_\ell$-elementary rings, and determine their covering numbers. Covers by right or two-sided ideals are also studied. This completely characterizes rings admitting finite covers by ideals. Our results generalize to finite covers of modules by submodules, and we determine all possible covering numbers.