We investigate the impact of noise and topology on opinion diversity in social networks. We do so by extending well-established models of opinion dynamics to a stochastic setting where agents are subject both to assimilative forces by their local social interactions, as well as to idiosyncratic factors preventing their population from reaching consensus. We model the latter to account for both scenarios where noise is entirely exogenous to peer influence and cases where it is instead endogenous, arising from the agents' desire to maintain some uniqueness in their opinions. We derive a general analytical expression for opinion diversity, which holds for any network and depends on the network's topology through its spectral properties alone. Using this expression, we find that opinion diversity decreases as communities and clusters are broken down. We test our predictions against data describing empirical influence networks between major news outlets and find that incorporating our measure in linear models for the sentiment expressed by such sources on a variety of topics yields a notable improvement in terms of explanatory power.