Let $P_t$ be the (Neumann) diffusion semigroup $P_t$ generated by a weighted Laplacian on a complete connected Riemannian manifold $M$ without boundary or with a convex boundary. It is well known that the Bakry-Emery curvature is bounded below by a positive constant $\ll>0$ if and only if $$W_p(μ_1P_t, μ_2P_t)\le \e^{-\ll t} W_p (μ_1,μ_2),\ \ t\ge 0, p\ge 1 $$ holds for all probability measures $μ_1$ and $μ_2$ on $M$, where $W_p$ is the $L^p$ Wasserstein distance induced by the Riemannian distance. In this paper, we prove the exponential contraction $$W_p(μ_1P_t, μ_2P_t)\le c\e^{-\ll t} W_p (μ_1,μ_2),\ \ p\ge 1, t\ge 0$$ for some constants $c,\ll>0$ for a class of diffusion semigroups with negative curvature where the constant $c$ is essentially larger than $1$. Similar results are derived for SDEs with multiplicative noise by using explicit conditions on the coefficients, which are new even for SDEs with additive noise.