We obtain general weak existence and stability results for stochastic convolution equations with jumps under mild regularity assumptions, allowing for non-Lipschitz coefficients and singular kernels. Our approach relies on weak convergence in $L^p$ spaces. The main tools are new a priori estimates on Sobolev-Slobodeckij norms of the solution, as well as a novel martingale problem that is equivalent to the original equation. This leads to generic approximation and stability theorems in the spirit of classical martingale problem theory. We also prove uniqueness and path regularity of solutions under additional hypotheses. To illustrate the applicability of our results, we consider scaling limits of nonlinear Hawkes processes and approximations of stochastic Volterra processes by Markovian semimartingales.