For time series with high temporal correlation, the empirical process converges rather slowly to its limiting distribution. Many statistics in change-point analysis, goodness-of-fit testing and uncertainty quantification admit a representation as functionals of the empirical process and therefore inherit its slow convergence. As a result, inference based on the asymptotic distribution of those quantities is significantly affected by relatively small sample sizes. We assess the quality of higher-order approximations of the empirical process by deriving the asymptotic distribution of the corresponding error terms. Based on the limiting distribution of the higher-order terms, we propose a novel approach to calculate confidence intervals for statistical quantities such as the median. In a simulation study, we compare coverage rates and lengths of these confidence intervals with those based on the asymptotic distribution of the empirical process and highlight some benefits of higher-order approximations of the empirical process.