While an important topic in practice, the estimation of the number of non-noise components in blind source separation has received little attention in the literature. Recently, two bootstrap-based techniques for estimating the dimension were proposed, and although very efficient, they suffer from the long computation times caused by the resampling. We approach the problem from a large sample viewpoint and develop an asymptotic test for the true dimension. Our test statistic based on second-order temporal information has a very simple limiting distribution under the null hypothesis and requires no parameters to estimate. Comparisons to the resampling-based estimates show that the asymptotic test provides comparable error rates with significantly faster computation time. An application to sound recording data is used to illustrate the method in practice.