Motivated by applications to mathematical biology, we study the averaging problem for slow-fast systems, {\em in the case in which the fast dynamics is a stochastic process with multiple invariant measures}. We consider both the case in which the fast process is decoupled from the slow process and the case in which the two components are fully coupled. We work in the setting in which the slow process evolves according to an Ordinary Differential Equation (ODE) and the fast process is a continuous time Markov Process with finite state space and show that, in this setting, the limiting (averaged) dynamics can be described as a random ODE (that is, an ODE with random coefficients.) Keywords. Multiscale methods, Processes with multiple equilibria, Averaging, Collective Navigation, Interacting Piecewise Deterministic Markov Processes.