Found in varied contexts from neurons to ants to fish, binary decision-making is one of the simplest forms of collective computation. In this process, information collected by individuals about an uncertain environment is accumulated to guide behavior at the aggregate scale. We study binary decision-making dynamics in networks responding to inputs with small signal-to-noise ratios, looking for quantitative measures of collectivity that control decision-making performance. We find that decision accuracy is controlled largely by three factors: the leading eigenvalue of the network adjacency matrix, the corresponding eigenvector's participation ratio, and distance from the corresponding symmetry-breaking bifurcation. This allows us to predict how decision-making performance scales in large networks based on their spectral properties. Specifically, we explore the effects of localization caused by the hierarchical assortative structure of a "rich club" topology. This gives insight into the tradeoffs involved in the higher-order structure found in living networks performing collective computations.