There is growing empirical evidence that spherical $k$-means clustering performs well at identifying groups of concomitant extremes in high dimensions, thereby leading to sparse models. We provide one of the first theoretical results supporting this approach, but also demonstrate some pitfalls. Furthermore, we show that an alternative cost function may be more appropriate for identifying concomitant extremes, and it results in a novel spherical $k$-principal-components clustering algorithm. Our main result establishes a broadly satisfied sufficient condition guaranteeing the success of this method, albeit in a rather basic setting. Finally, we illustrate in simulations that $k$-principal-components outperforms $k$-means in the difficult case of weak asymptotic dependence within the groups.