We consider additive functionals of stationary Markov processes and show that under Kipnis-Varadhan type conditions they converge in rough path topology to a Stratonovich Brownian motion, with a correction to the Levy area that can be described in terms of the asymmetry (non-reversibility) of the underlying Markov process. We apply this abstract result to three model problems: First we study random walks with random conductances under the annealed law. If we consider the Ito rough path, then we see a correction to the iterated integrals even though the underlying Markov process is reversible. If we consider the Stratonovich rough path, then there is no correction. The second example is a non-reversible Ornstein-Uhlenbeck process, while the last example is a diffusion in a periodic environment. As a technical step we prove an estimate for the p-variation of stochastic integrals with respect to martingales that can be viewed as an extension of the rough path Burkholder-Davis-Gundy inequalities for local martingale rough paths of Friz et al to the case where only the integrator is a local martingale.