In the present paper we obtain sufficient conditions for the existence of equivalent martingale measures for Lévy-driven moving averages and other non-Markovian jump processes. The conditions that we obtain are, under mild assumptions, also necessary. For instance, this is the case for moving averages driven by an $α$-stable Lévy process with $α\in (1,2]$. Our proofs rely on various techniques for showing the martingale property of stochastic exponentials.