In this article, we provide detailed analysis of the long-time behavior of the underdamped Langevin dynamics. We first provide a necessary condition guaranteeing that the zero-noise dynamical system converges to its unique attractor. We also observed that this condition is sharp for a large class of linear models. We then prove the so-called cut-off phenomenon in the small-noise regime under this condition. This result provides the precise asymptotics of the mixing time of the process and of the distance between the distribution of the process and its stationary measure. The main difficulty of this work relies on the degeneracy of its infinitesimal generator which is not elliptic, thus requiring a new set of methods.