Stochastic models for collections of interacting populations have crucial roles in scientific fields such as epidemiology and ecology, yet the standard approach to extending an ordinary differential equation model to a Markov chain does not have sufficient flexibility in the mean-variance relationship to match data. To handle that, over-dispersed Markov chains have previously been constructed using gamma white noise on the rates. We develop new approaches using Dirichlet noise to construct collections of independent or dependent noise processes. This permits the modeling of high-frequency variation in transition rates both within and between the populations under study. Our theory is developed in a general framework of time-inhomogeneous Markov processes equipped with a graphical structure, for which ecological and epidemiological models provide motivating examples. We demonstrate our approach on a widely analyzed measles dataset, adding Dirichlet noise to a classical SEIR (Susceptible-Exposed-Infected-Recovered) model. Our methodology shows improved statistical fit measured by log-likelihood and provides new insights into the dynamics of this biological system.