We study statistical inference of the drift parameters for the Volterra Ornstein-Uhlenbeck process on R in the ergodic regime. For continuous-time observations, we derive the corresponding maximum likelihood estimators and show that they are strongly consistent and asymptotically normal locally uniformly in the parameters. For the case of discrete high-frequency observations, we prove similar results by discretization of the continuous-time maximum likelihood estimator. Finally, for discrete low-frequency observations, we show that the method of moments is consistent. Our proofs are crucially based on the law of large numbers.