We study the Langevin dynamics for spherical $p$-spin models, focusing on the short time regime described by the Cugliandolo-Kurchan equations. Confirming a prediction of [Cugliandolo-Kurchan, Phys. Rev. Lett. 1993], we show the asymptotic energy achieved is exactly $E_{\infty}(p)=2\sqrt{\frac{p-1}{p}}$ in the low temperature limit. The upper bound uses hardness results for Lipschitz optimization algorithms and applies for all temperatures. For the lower bound, we prove the dynamics reaches and stays above the lowest energy of any approximate local maximum. In fact the latter behavior holds for any Hamiltonian obeying natural smoothness estimates, even with disorder-dependent initialization and on exponential time-scales.