We show that any proper coloring of a Kneser graph $KG_{n,k}$ with $n-2k+2$ colors contains a trivial color (i.e., a color consisting of sets that all contain a fixed element), provided $n>(2+\varepsilon)k^2$, where $\varepsilon\to 0$ as $k\to \infty$. This bound is essentially tight. This is a consequence of a more general result on the minimum number of non-trivial colors needed to properly color $KG_{n,k}$.