The formation of collective opinion is a complex phenomenon that results from the combined effects of mass media exposure and social influence between individuals. The present work introduces a model of opinion formation specifically designed to address risk judgments, such as attitudes towards climate change, terrorist threats, or children vaccination. The model assumes that people collect risk information from the media environment and exchange them locally with other individuals. Even though individuals are initially exposed to the same sample of information, the model predicts the emergence of opinion polarization and clustering. In particular, numerical simulations highlight two crucial factors that determine the collective outcome: the propensity of individuals to search for independent information, and the strength of social influence. This work provides a quantitative framework to anticipate and manage how the public responds to a given risk, and could help understanding the systemic amplification of fears and worries, or the underestimation of real dangers.