Atar, Chowdhary and Dupuis have recently exhibited a variational formula for exponential integrals of bounded measurable functions in terms of Rényi divergences. We develop a variational characterization of the Rényi divergences between two probability distributions on a measurable sace in terms of relative entropies. When combined with the elementary variational formula for exponential integrals of bounded measurable functions in terms of relative entropy, this yields the variational formula of Atar, Chowdhary and Dupuis as a corollary. We also develop an analogous variational characterization of the Rényi divergence rates between two stationary finite state Markov chains in terms of relative entropy rates. When combined with Varadhan's variational characterization of the spectral radius of square matrices with nonnegative entries in terms of relative entropy, this yields an analog of the variational formula of Atar, Chowdary and Dupuis in the framework of finite state Markov chains.