From the formation of ice in small clusters of water molecules to the mass raids of army ant colonies, the emergent behavior of collectives depends critically on their size. At the same time, common wisdom holds that such behaviors are robust to the loss of individuals. This tension points to the need for a more systematic study of how number influences collective behavior. We initiate this study by focusing on collective behaviors that change abruptly at certain critical numbers of individuals. We show that a subtle modification of standard bifurcation analysis identifies such critical numbers, including those associated with discreteness- and noise-induced transitions. By treating them as instances of the same phenomenon, we show that critical numbers across physical scales and scientific domains commonly arise from competing feedbacks that scale differently with number. We then use this idea to find overlooked critical numbers in past studies of collective behavior and explore the implications for their conclusions. In particular, we highlight how deterministic approximations of stochastic models can fail near critical numbers. We close by distinguishing these qualitative changes from density-dependent phase transitions and by discussing how our approach could generalize to broader classes of collective behaviors.