We study the problem of exact support recovery based on noisy observations and present Refined Least Squares (RLS). Given a set of noisy measurement $$ \myvec{y} = \myvec{X}\myvecθ^* + \myvecω,$$ and $\myvec{X} \in \mathbb{R}^{N \times D}$ which is a (known) Gaussian matrix and $\myvecω \in \mathbb{R}^N$ is an (unknown) Gaussian noise vector, our goal is to recover the support of the (unknown) sparse vector $\myvecθ^* \in \left\{-1,0,1\right\}^D$. To recover the support of the $\myvecθ^*$ we use an average of multiple least squares solutions, each computed based on a subset of the full set of equations. The support is estimated by identifying the most significant coefficients of the average least squares solution. We demonstrate that in a wide variety of settings our method outperforms state-of-the-art support recovery algorithms.