A general framework for obtaining exact transition rate matrices for stochastic systems on networks is presented and applied to many well-known compartmental models of epidemiology. The state of the population is described as a vector in the tensor product space of $N$ individual probability vector spaces, whose dimension equals the number of compartments of the epidemiological model $n_c$. The transition rate matrix for the $n_c^N$-dimensional Markov chain is obtained by taking suitable linear combinations of tensor products of $n_c$-dimensional matrices. The resulting transition rate matrix is a sum over bilocal linear operators, which gives insight in the microscopic dynamics of the system. The more familiar and non-linear node-based mean-field approximations are recovered by restricting the exact models to uncorrelated (separable) states. We show how the exact transition rate matrix for the susceptible-infected (SI) model can be used to find analytic solutions for SI outbreaks on trees and the cycle graph for finite $N$.