We consider approximation of a Gaussian distribution with a mixture of homoscedastic Gaussians of smaller variance. The solution is obtained by minimising the $L^2$ norm between the original Gaussian and the mixture, which is parameterised to reduce the complexity of the optimisation problem. The developed technique is straightforward, sufficiently robust and yields Gaussian Mixtures that rapidly approach the original function as the number of mixands is increased. The proposed solution is examined for multiple special cases of input parameters resulting in further simplifications. Extension of the proposed method for approximating non-Gaussian distributions is discussed.