Given a regular (connected) graph $Γ=(X,E)$ with adjacency matrix $A$, $d+1$ distinct eigenvalues, and diameter $D$, we give a characterization of when its distance matrix $A_D$ is a polynomial in $A$, in terms of the adjacency spectrum of $Γ$ and the arithmetic (or harmonic) mean of the numbers of vertices at distance $\le D-1$ of every vertex. The same results is proved for any graph by using its Laplacian matrix $L$ and corresponding spectrum. When $D=d$ we reobtain the spectral excess theorem characterizing distance-regular graphs.