We study the set of $k$-abelian critical exponents of all Sturmian words. It has been proven that in the case $k = 1$ this set coincides with the Lagrange spectrum. Thus the sets obtained when $k > 1$ can be viewed as generalized Lagrange spectra. We characterize these generalized spectra in terms of the usual Lagrange spectrum and prove that when $k > 1$ the spectrum is a dense non-closed set. This is in contrast with the case $k = 1$, where the spectrum is a closed set containing a discrete part and a half-line. We describe explicitly the least accumulation points of the generalized spectra. Our geometric approach allows the study of $k$-abelian powers in Sturmian words by means of continued fractions.