A growing statistical literature focuses on causal inference in the context of experiments where the target of inference is the average treatment effect in a finite population and random assignment determines which subjects are allocated to one of the experimental conditions. In this framework, variances of average treatment effect estimators remain unidentified because they depend on the covariance between treated and untreated potential outcomes, which are never jointly observed. Conventional variance estimators are upwardly biased. Aronow, Green and Lee [Ann. Statist. 42(3): 850-871 (June 2014)] provide an estimator for the variance of the difference-in-means estimator that is asymptotically sharp. In practice, researchers often use some form of covariate adjustment, such as linear regression, when estimating the average treatment effect. Adapting propositions from empirical process theory, we extend the result in (Aronow et al., 2014), providing asymptotically sharp variance bounds for general regression adjustment. We apply these results to linear regression adjustment and show benefits both in a simulation and in three empirical applications drawn from different disciplines.