This paper investigates performance guarantees on coverage-based ergodic exploration methods in environments containing disturbances. Ergodic exploration methods generate trajectories for autonomous robots such that time spent in each area of the exploration space is proportional to the utility of exploring in the area. We find that it is possible to use techniques from reachability analysis to solve for optimal controllers that guarantee ergodic coverage and are robust against disturbances. We formulate ergodic search as a differential game between the controller optimizing for ergodicity and an external disturbance, and we derive the reachability equations for ergodic search using an extended-state Bolza-form transform of the ergodic problem. Contributions include the computation of a continuous value function for the ergodic exploration problem and the derivation of a controller that provides guarantees for coverage under disturbances. Our approach leverages neural-network-based methods to solve the reachability equations; we also construct a robust model-predictive controller for comparison. Simulated and experimental results demonstrate the efficacy of our approach for generating robust ergodic trajectories for search and exploration on a 1D system with an external disturbance force.