One of the key ingredients to successfully apply Stein's method for distributional approximation are solutions to the Stein equations and their derivatives. Using Barbour's generator approach, one can solve for the solutions to the Stein equation in terms of the semi-group of a Markov process, which is typically a diffusion process if it is a continuous distribution. For an arbitrary diffusion it can a difficult task to evaluate the semi-group and its derivatives. In this paper, for polynomial test functions, instead of calculating the semi-group of a diffusion, via a duality argument, we instead utilise the semi-group of a much simpler Markov jump process. This approach yields a new method for explicitly solving for the solutions of Stein equations for diffusion processes. We present both the general idea of the approach and examples for both univariate and multivariate distributions.