The dynamics of many-body systems can often be captured in terms of only a few relevant variables. Mathematical and numerical approaches exist to identify these variables by exploiting a separation of time scales between slow relevant and fast irrelevant variables, but such a separation of scales is not always obvious or even available. In this work, we introduce an information-theoretic framework for dimensionality reduction in dynamical systems that bypasses this limitation by instead identifying relevant variables based on how predictive they are of the system's future. To do so, we mathematically formalize the intuition that model reduction is about keeping "relevant" information while throwing away "irrelevant" information. We characterize the solution of the resulting optimization problem and prove that it reduces to standard approaches when a separation of time scales is indeed present in the dynamics. Importantly, we find that within this framework, the problems of identifying relevant variables and identifying their effective dynamics decouple and may be solved separately. This makes the method tractable in practice and enables us to derive dimensionally-reduced variables from data with neural networks. Combined with existing equation learning methods, the procedure introduced in this work reveals the dynamical rules governing the system's evolution in a data-driven manner. We illustrate these tools in diverse settings including simulated chaotic systems, uncurated satellite recordings of atmospheric fluid flows, and experimental videos of cyanobacteria colonies in which we discover an emergent synchronization order parameter.