We study how memory impacts passages at the origin for a so-called elephant random walk in the diffusive regime. We observe that the number of zeros always grows asymptotically like the square root of the time, despite the fact that, depending on the memory parameter, first return times to $0$ may have a finite expectation or a fat tail with exponent less than $1/2$. We resolve this apparent paradox by recasting the questions in the framework of scaling limits for Markov chains and self-similar Markov processes.