Recently, sharp results concerning the critical points of the Hamiltonian of the $p$-spin spherical spin glass model have been obtained by means of moments computations. In particular, these moments computations allow for the evaluation of the leading term of the ground-state, i.e., of the global minimum. In this paper, we study the extremal point process of critical points - that is, the point process associated to all critical values in the vicinity of the ground-state. We show that the latter converges in distribution to a Poisson point process of exponential intensity. In particular, we identify the correct centering of the ground-state and prove the convergence in distribution of the centered minimum to a (minus) Gumbel variable. These results are identical to what one obtains for a sequence of i.i.d variables, correctly normalized; namely, we show that the model is in the universality class of REM.