We consider the estimation of integrated covariance (ICV) matrices of high dimensional diffusion processes based on high frequency observations. We start by studying the most commonly used estimator, the realized covariance (RCV) matrix. We show that in the high dimensional case when the dimension $p$ and the observation frequency $n$ grow in the same rate, the limiting spectral distribution (LSD) of RCV depends on the covolatility process not only through the targeting ICV, but also on how the covolatility process varies in time. We establish a Marčenko--Pastur type theorem for weighted sample covariance matrices, based on which we obtain a Marčenko--Pastur type theorem for RCV for a class $\mathcal{C}$ of diffusion processes. The results explicitly demonstrate how the time variability of the covolatility process affects the LSD of RCV. We further propose an alternative estimator, the time-variation adjusted realized covariance (TVARCV) matrix. We show that for processes in class $\mathcal {C}$, the TVARCV possesses the desirable property that its LSD depends solely on that of the targeting ICV through the Marčenko--Pastur equation, and hence, in particular, the TVARCV can be used to recover the empirical spectral distribution of the ICV by using existing algorithms.