We propose a new estimator of high-dimensional spectral density matrices, called UNshrunk ALgebraic Spectral Estimator (UNALSE), under the assumption of an underlying low rank plus sparse structure, as typically assumed in dynamic factor models. The UNALSE is computed by minimizing a quadratic loss under a nuclear norm plus $l_1$ norm constraint to control the latent rank and the residual sparsity pattern. The loss function requires as input the classical smoothed periodogram estimator and two threshold parameters, the choice of which is thoroughly discussed. We prove consistency of UNALSE as both the dimension $p$ and the sample size $T$ diverge to infinity, as well as algebraic consistency, i.e., the recovery of latent rank and residual sparsity pattern with probability one. The finite sample properties of UNALSE are studied by means of an extended simulation exercise as well as an empirical analysis of US macroeconomic data.