It is well known that a non-negative definite polynomial matrix (a polynomial Gramian) $G(t)$ can be written as a product of its polynomial spectral factors, $G(t) = X(t)^H X(t)$. In this paper, we give a new algebraic characterization of spectral factors when $G(t)$ is real-valued. The key idea is to construct a representation set that is in bijection with the set of real polynomial Gramians. We use the derived characterization to identify the set of all complex polynomial matrices that generate real-valued Gramians, and we formulate a conjecture that typical rank-deficient real polynomial Gramians have real spectral factors.