For a sample of $n$ independent identically distributed $p$-dimensional centered random vectors with covariance matrix $\mathbfΣ_n$ let $\tilde{\mathbf{S}}_n$ denote the usual sample covariance (centered by the mean) and $\mathbf{S}_n$ the non-centered sample covariance matrix (i.e. the matrix of second moment estimates), where $p> n$. In this paper, we provide the limiting spectral distribution and central limit theorem for linear spectral statistics of the Moore-Penrose inverse of $\mathbf{S}_n$ and $\tilde{\mathbf{S}}_n$. We consider the large dimensional asymptotics when the number of variables $p\rightarrow\infty$ and the sample size $n\rightarrow\infty$ such that $p/n\rightarrow c\in (1, +\infty)$. We present a Marchenko-Pastur law for both types of matrices, which shows that the limiting spectral distributions for both sample covariance matrices are the same. On the other hand, we demonstrate that the asymptotic distribution of linear spectral statistics of the Moore-Penrose inverse of $\tilde{\mathbf{S}}_n$ differs in the mean from that of $\mathbf{S}_n$.