We study the extremality properties of rays generated by tautological Chern classes of matroids in the polyhedral nef cones of the permutohedral variety. Our main result provides necessary and sufficient conditions a matroid has to meet for its tautological Chern classes to generate extremal rays. Our approach yields a new and large family of extremal rays that interpolates between two previously known classes coming from rank functions and Bergman classes of matroids. We further discuss how this theorem extends to the stellahedral variety through augmented tautological Chern classes.