We provide a refined explicit estimate of exponential decay rate of underdamped Langevin dynamics in $L^2$ distance, based on a framework developed in [1]. To achieve this, we first prove a Poincaré-type inequality with Gibbs measure in space and Gaussian measure in momentum. Our estimate provides a more explicit and simpler expression of decay rate; moreover, when the potential is convex with Poincaré constant $m \ll 1$, our estimate shows the decay rate of $O(\sqrt{m})$ after optimizing the choice of friction coefficient, which is much faster than $m$ for the overdamped Langevi dynamics.