Stochastic systems with memory naturally appear in life science, economy, and finance. We take the modelling point of view of stochastic functional delay equations and we study these structures when the driving noises admit jumps. Our results concern existence and uniqueness of strong solutions, estimates for the moments and the fundamental tools of calculus, such as the Itô formula. We study the robustness of the solution to the change of noises. Specifically, we consider the noises with infinite activity jumps versus an adequately corrected Gaussian noise. The study is presented in two different frameworks: we work with random variables in infinite dimensions, where the values are considered either in an appropriate $L^p$-type space or in the space of càdlàg paths. The choice of the value space is crucial from the modelling point of view as the different settings allow for the treatment of different models of memory or delay. Our techniques involve tools of infinite dimensional calculus and the stochastic calculus via regularization.