When learning interpretable latent structures using model-based approaches, even small deviations from modeling assumptions can lead to inferential results that are not mechanistically meaningful. In this work, we consider latent structures that consist of $K_o$ mechanistic processes, where $K_o$ is unknown. When the model is misspecified, likelihood-based model selection methods can substantially overestimate $K_o$ while more robust nonparametric methods can be overly conservative. Hence, there is a need for approaches that combine the sensitivity of likelihood-based methods with the robustness of nonparametric ones. We formalize this objective in terms of a robust model selection consistency property, which is based on a component-level discrepancy measure that captures the mechanistic structure of the model. We then propose the accumulated cutoff discrepancy criterion (ACDC), which leverages plug-in estimates of component-level discrepancies. To apply ACDC, we develop mechanistically meaningful component-level discrepancies for a general class of latent variable models that includes unsupervised and supervised variants of probabilistic matrix factorization and mixture modeling. We show that ACDC is robustly consistent when applied to unsupervised matrix factorization and mixture models. Numerical results demonstrate that in practice our approach reliably identifies a mechanistically meaningful number of latent processes in numerous illustrative applications, outperforming existing methods.