We consider coupled slow-fast stochastic processes, where the averaged slow motion is given by a two-dimensional Hamiltonian system with multiple critical points. On a proper time scale, the evolution of the first integral converges to a diffusion process on the corresponding Reeb graph, with certain gluing conditions specified at the interior vertices, as in the case of additive white noise perturbations of Hamiltonian systems considered by M. Freidlin and A. Wentzell. The current paper provides the first result where the motion on a graph and the corresponding gluing conditions appear due to the averaging of a slow-fast system, with a Hamiltonian structure, on a large time scale. The result allows one to consider, for instance, long-time diffusion approximation for an oscillator with a potential with more than one well.