We consider high-dimensional Wishart matrices $d^{-1}\mathcal{X}_{n,d}\mathcal{X}_{n,d}^T$, associated with a rectangular random matrix $\mathcal{X}_{n,d}$ of size $n\times d$ whose entries are jointly Gaussian and correlated. Even if we will consider the case of overall correlation among the entries of $\mathcal{X}_{n,d}$, our main focus is on the case where the rows of $\mathcal{X}_{n,d}$ are independent copies of a $n$-dimensional stationary centered Gaussian vector of correlation function $s$. When $s$ belongs to $\ell^{4/3}(\mathbb{Z})$, we show that a proper normalization of $d^{-1}\mathcal{X}_{n,d}\mathcal{X}_{n,d}^T$ is close in Wasserstein distance to the corresponding Gaussian ensemble as long as $d$ is much larger than $n^3$, thus recovering the main finding of [3,9] and extending it to a larger class of matrices. We also investigate the case where $s$ is the correlation function associated with the fractional Brownian noise of parameter $H$. This example is very rich, as it gives rise to a great variety of phenomena with very different natures, depending on how $H$ is located with respect to $1/2$, $5/8$ and $3/4$. Notably, when $H>3/4$, our study highlights a new probabilistic object, which we have decided to call the Rosenblatt-Wishart matrix. Our approach crucially relies on the fact that the entries of the Wishart matrices we are dealing with are double Wiener-Itô integrals, allowing us to make use of multivariate bounds arising from the Malliavin-Stein method and related ideas. To conclude the paper, we analyze the situation where the row-independence assumption is relaxed and we also look at the setting of random $p$-tensors ($p\geq 3$), a natural extension of Wishart matrices.