In the political arena of social platforms, opposing factions of varying sizes show asymmetrical patterns, and elites and masses within these groups have divergent motivations and influence,challenging simplistic views of polarization. Yet, existing methods for quantifying polarization reduce division to a single value, assuming uniform distribution of polarization online. While this approach can confirm the observed increase in political polarization in many societies, it overlooks complexities that could explain this phenomenon. Notably, opposing groups can have unequal impacts on polarization, and the literature shows division between elites and the masses is a critical factor to consider. We propose a method to decompose existing polarization measures in order to quantify the role of groups, determined by these distinct hierarchies, in the total polarization value. We applied this method to polarized topics in the Finnish Twittersphere surrounding the 2019 and 2023parliamentary elections. Our analysis reveals two key insights: 1) The impact of opposing groups on observed polarization is rarely balanced, and 2) while elites strongly contribute to structural polarization and consistently display greater alignment across various topics, the masses have also recently experienced a surge in issue alignment, a stronger form of polarization. Our findings suggest that the masses may not be as immune to an increasingly polarized environment as previously thought. This research provides a more nuanced understanding of polarization dynamics, offering potential insights into its underlying mechanisms and evolution