For reversible Markov chains on finite state spaces, we show that the modified log-Sobolev inequality (MLSI) can be upgraded to a log-Sobolev inequality (LSI) at the surprisingly low cost of degrading the associated constant by $\log (1/p)$, where $p$ is the minimum non-zero transition probability. We illustrate this by providing the first log-Sobolev estimate for Zero-Range processes on arbitrary graphs. As another application, we determine the modified log-Sobolev constant of the Lamplighter chain on all bounded-degree graphs, and use it to provide negative answers to two open questions by Montenegro and Tetali (2006) and Hermon and Peres (2018). Our proof builds upon the `regularization trick' recently introduced by the last two authors.