In this article, exponential contraction in Wasserstein distance for heat semigroups of diffusion processes on Riemannian manifolds is established under curvature conditions where Ricci curvature is not necessarily required to be non-negative. Compared to the results of Wang (2016), we focus on explicit estimates for the exponential contraction rate. Moreover, we show that our results extend to manifolds evolving under a geometric flow. As application, for the time-inhomogeneous semigroups, we obtain a gradient estimate with an exponential contraction rate under weak curvature conditions, as well as uniqueness of the corresponding evolution system of measures.