We prove that for a random walk on the real line whose increments have zero mean and are either integer-valued or spread out (i.e. the distributions of the steps of the walk are eventually non-singular), the Markov chain of overshoots above a fixed level converges in total variation to its stationary distribution. We find the explicit form of this distribution heuristically and then prove its invariance using a time-reversal argument. If, in addition, the increments of the walk are in the domain of attraction of a non-one-sided $α$-stable law with index $α\in(1,2)$ (resp. have finite variance), we establish geometric (resp. uniform) ergodicity for the Markov chain of overshoots. All the convergence results above are also valid for the Markov chain obtained by sampling the walk at the entrance times into an interval.