We analyse the Hessian of the Thouless-Anderson-Palmer (TAP) free energy for the Sherrington-Kirkpatrick model, below the de Almeida-Thouless line, evaluated in Bolthausen's approximate solutions of the TAP equations. We show that the empirical spectral distribution weakly converges to a measure with negative support below the AT line, an that the support includes zero on the AT line. In this ``macroscopic'' sense, TAP free energy is concave in the order parameter of the theory, i.e. the random spin-magnetisations. This proves a spectral interpretation of the AT line. However, for specific magnetizations, the Hessian of the TAP free energy can have positive outlier eigenvalues. The question whether such outliers may also occur close to the TAP solutions is left open. In a simplified setting where the magnetizations are independent of the disorder, we prove that Plefka's second condition is equivalent to all eigenvalues being negative.