Motivated by the dynamics of cultural change and diversity, we generalize the three-species constrained voter model on a complete graph introduced in [J. Phys. A 37, 8479 (2004)]. In this opinion dynamics model, a population of size N is composed of "leftists" and "rightists" that interact with "centrists": a leftist and centrist can both become leftists with rate (1+q)/2 or centrists with rate (1-q)/2 (and similarly for rightists and centrists), where q denotes the bias towards extremism (q>0) or centrism (q<0). This system admits three absorbing fixed points and a "polarization" line along which a frozen mixture of leftists and rightists coexist. In the realm of Fokker-Planck equation, and using a mapping onto a population genetics model, we compute the fixation probability of ending in every absorbing state and the mean times for these events. We therefore show, especially in the limit of weak bias and large population size when |q|~1/N and N>>1, how fluctuations alter the mean field predictions: polarization is likely when q>0, but there is always a finite probability to reach a consensus; the opposite happens when q<0. Our findings are corroborated by stochastic simulations.