In this work, we take a step towards understanding overdamped Langevin dynamics for the minimization of a general class of objective functions $\mathcal{L}$. We establish well-posedness and regularity of the law $ρ_t$ of the process through novel a priori estimates, and, very importantly, we characterize the large-time behavior of $ρ_t$ under truly minimal assumptions on $\mathcal{L}$. In the case of integrable Gibbs density, the law converges to the normalized Gibbs measure. In the non-integrable case, we prove that the law diffuses. The rate of convergence is $\mathcal{O}(1/t)$. Under a Polyak-Lojasiewicz (PL) condition on $\mathcal{L}$, we also derive sharp exponential contractivity results toward the set of global minimizers. Combining these results we provide the first systematic convergence analysis of Langevin dynamics under PL conditions in non-integrable Gibbs settings: a first phase of exponential in time contraction toward the set of minimizers and then a large-time exploration over it with rate $\mathcal{O}(1/t)$.