Measuring culture and its dynamics through surveys has important limitations, but the emerging field of computational social science allows us to overcome them by analyzing large-scale datasets. In this article, we study cultural dynamics through the votes in the Eurovision song contest, which are decided by a crowd-based scheme in which viewers vote through mobile phone messages. Taking into account asymmetries and imperfect perception of culture, we measure cultural relations among European countries in terms of cultural affinity. We propose the Friend-or-Foe coefficient, a metric to measure voting biases among participants of a Eurovision contest. We validate how this metric represents cultural affinity through its relation with known cultural distances, and through numerical analysis of biased Eurovision contests. We apply this metric to the historical set of Eurovision contests from 1975 to 2012, finding new patterns of stronger modularity than using votes alone. Furthermore, we define a measure of polarization that, when applied to empirical data, shows a sharp increase within EU countries during 2010 and 2011. We empirically validate the relation between this polarization and economic indicators in the EU, showing how political decisions influence both the economy and the way citizens relate to the culture of other EU members.