Colored graphical models provide a parsimonious approach to modeling high-dimensional data by exploiting symmetries in the model parameters. In this work, we introduce the notion of coloring for extremal graphical models on multivariate Pareto distributions, a natural class of limiting distributions for threshold exceedances. Thanks to a stability property of the multivariate Pareto distributions, colored extremal tree models can be defined fully nonparametrically. For more general graphs, the parametric family of Hüsler--Reiss distributions allows for two alternative approaches to colored graphical models. We study both model classes and introduce statistical methodology for parameter estimation. It turns out that for Hüsler--Reiss tree models the different definitions of colored graphical models coincide. In addition, we show a general parametric description of extremal conditional independence statements for Hüsler--Reiss distributions. Finally, we demonstrate that our methodology outperforms existing approaches on a real data set.