We consider a class of sparse random matrices which includes the adjacency matrix of the Erdős-Rényi graph $\mathcal{G}(N,p)$. We show that if $N^{\varepsilon} \leq Np \leq N^{1/3-\varepsilon}$ then all nontrivial eigenvalues away from 0 have asymptotically Gaussian fluctuations. These fluctuations are governed by a single random variable, which has the interpretation of the total degree of the graph. This extends the result [19] on the fluctuations of the extreme eigenvalues from $Np \geq N^{2/9 + \varepsilon}$ down to the optimal scale $Np \geq N^{\varepsilon}$. The main technical achievement of our proof is a rigidity bound of accuracy $N^{-1/2-\varepsilon} \, (Np)^{-1/2}$ for the extreme eigenvalues, which avoids the $(Np)^{-1}$-expansions from [9,19,24]. Our result is the last missing piece, added to [8, 12, 19, 24], of a complete description of the eigenvalue fluctuations of sparse random matrices for $Np \geq N^{\varepsilon}$.