We study averaging for Stochastic Differential Equations (SDEs) and Poisson equations. We succeed in obtaining a uniform in time (UiT) averaging result, with a rate, for fully coupled SDE models with super-linearly growing coefficients. This is the main result of this paper and is, to the best of our knowledge, the first UiT multiscale result with a rate. Very few UiT averaging results exist in the literature, and they almost exclusively apply to multiscale systems of Ordinary Differential Equations. Among these few, none of those we are aware of comes with a rate of convergence. The UiT nature of this result and the rate of convergence given by the main theorem, make it important as theoretical underpinning for a range of applications, such as applications to statistical methodology, molecular dynamics etc. Key to obtaining both our UiT averaging result and to enable dealing with the super-linear growth of the coefficients is conquering exponential decay in time of the space-derivatives of appropriate Markov semigroups. We refer to this property as being Strongly Exponentially Stable (SES). The analytic approach to proving averaging results we take requires studying a family of Poisson problems associated with the generator of the (fast component of the) SDE dynamics. The study of Poisson equations in non-compact state space is notoriously difficult, with current literature mostly covering the case when the coefficients of the Partial Differential Equation (PDE) are either bounded or satisfy linear growth assumptions. In this paper we treat Poisson equations on non-compact state spaces for coefficients that can grow super-linearly. We demonstrate how SES can be employed not only to prove the UiT result for the slow-fast system but also to overcome some of the technical hurdles in the analysis of Poisson problems, which is of independent interest as well.