For non-autonomous linear stochastic differential equations (SDEs), we establish that the top Lyapunov exponent is continuous if the coefficients "almost" uniformly converge. For autonomous SDEs, assuming the existence of invariant measures and the convergence of coefficients and their derivatives in pointwise sense, we get the continuity of all Lyapunov exponents. Furthermore, we demonstrate that for autonomous SDEs with strict monotonicity condition, all Lyapunov exponents are Lipschitz continuous with respect to the coefficients under the $L^{p,p}$ norm ($p>2$). Similarly, the Hölder continuity of Lyapunov exponents holds under weaker regularity conditions. It seems that the continuity of Lyapunov exponents has not been studied for SDEs so far, in spite that there are many results in this direction for discrete-time dynamical systems.