We consider the Sherrington-Kirkpatrick model with no external field and inverse temperature $β<1$ and prove that the expected operator norm of the covariance matrix of the Gibbs measure is bounded by a constant depending only on $β$. This answers an open question raised by Talagrand, who proved a bound of $C(β) (\log n)^8$. Our result follows by establishing an approximate formula for the covariance matrix which we obtain by differentiating the TAP equations and then optimally controlling the associated error terms. We complement this result by showing diverging lower bounds on the operator norm, both at the critical and low temperatures.