Online platforms play a relevant role in the creation and diffusion of false or misleading news. Concerningly, the COVID-19 pandemic is shaping a communication network - barely considered in the literature - which reflects the emergence of collective attention towards a topic that rapidly gained universal interest. Here, we characterize the dynamics of this network on Twitter, analyzing how unreliable content distributes among its users. We find that a minority of accounts is responsible for the majority of the misinformation circulating online, and identify two categories of users: a few active ones, playing the role of "creators", and a majority playing the role of "consumers". The relative proportion of these groups ($\approx$14% creators - 86% consumers) appears stable over time: Consumers are mostly exposed to the opinions of a vocal minority of creators, that could be mistakenly understood as of representative of the majority of users. The corresponding pressure from a perceived majority is identified as a potential driver of the ongoing COVID-19 infodemic.