Modern recordings of neural activity provide diverse observations of neurons across brain areas, conditions, and subjects; presenting an exciting opportunity to reveal the fundamentals of brain-wide dynamics. Current analysis methods often fail to harness the richness of such data, as they provide either uninterpretable representations or oversimplify models (e.g., by assuming stationary dynamics). Here, instead of regarding asynchronous neural recordings that lack alignment in neural identity or brain areas as a limitation, we leverage these diverse views into the brain to learn a unified model of neural dynamics. We assume that brain activity is driven by multiple hidden global sub-circuits. These sub-circuits represent global basis interactions between neural ensembles -- functional groups of neurons -- such that the time-varying decomposition of these circuits defines how the ensembles' interactions evolve over time non-stationarily. We discover the neural ensembles underlying non-simultaneous observations, along with their non-stationary evolving interactions, with our new model, CREIMBO. CREIMBO identifies the hidden composition of per-session neural ensembles through graph-driven dictionary learning and models the ensemble dynamics on a low-dimensional manifold spanned by a sparse time-varying composition of the global sub-circuits. Thus, CREIMBO disentangles overlapping temporal neural processes while preserving interpretability due to the use of a shared underlying sub-circuit basis. Moreover, CREIMBO distinguishes session-specific computations from global (session-invariant) ones by identifying session covariates and variations in sub-circuit activations. We demonstrate CREIMBO's ability to recover true components in synthetic data, and uncover meaningful brain dynamics including cross-subject neural mechanisms and inter- vs. intra-region dynamical motifs.