Regime switching processes have proved to be indispensable in the modeling of various phenomena, allowing model parameters that traditionally were considered to be constant to fluctuate in a Markovian manner in line with empirical findings. We study diffusion processes of Ornstein-Uhlenbeck type where the drift and diffusion coefficients $a$ and $b$ are functions of a Markov process with a stationary distribution $π$ on a countable state space. Exact long time behavior is determined for the three regimes corresponding to the expected drift: $E_πa(\cdot)>0,=0,<0$, respectively. Alongside we provide exact time limit results for integrals of form $\int_{0}^{t}b^{2}(X_{s})e^{-2\int_{s}^{t}a(X_{r})dr}ds$ for the three different regimes. Finally, we demonstrate natural applications of the findings in terms of Cox-Ingersoll-Ross diffusion and deterministic SIS epidemic models in Markovian environments. Exact long time behaviors are naturally expressed in terms of solutions to the well-studied fixed-point equation in law $X\stackrel{d}{=}AX+B$ with $X \indep (A,B)$.