Suppose $ E$ is a space with a null-recurrent Markov kernel $ P$. Furthermore, suppose there are infinite particles with variable weights on $ E$ performing a random walk following $ P$. Let $ X_{t}$ be a weighted functional of the position of particles at time $ t$. Under some conditions on the initial distribution of the particles the process $ (X_{t})$ is stationary over time. Non-Gaussian infinitely divisible (ID) distributions turn out to be natural candidates for the initial distribution and then the process $ (X_{t})$ is ID. We prove a functional large and moderate deviation principle for the partial sums of the process $ (X_{t})$. The recurrence of the Markov Kernel $ P$ induces long memory in the process $ (X_{t})$ and that is reflected in the large deviation principle. It has been observed in certain short memory processes that the large deviation principle is very similar to that of an i.i.d. sequence. Whereas, if the process is long range dependent the large deviations change dramatically. We show that a similar phenomenon is observed for infinitely divisible processes driven by Markov chains. Processes of the form of $ (X_{t})$ gives us a rich class of non-Gaussian long memory models which may be useful in practice.