This work resolves the following question in non-Euclidean statistics: Is it possible to consistently estimate the Fréchet mean set of an unknown population distribution, with respect to the Hausdorff metric, when given access to independent identically-distributed samples? Our affirmative answer is based on a careful analysis of the "relaxed empirical Fréchet mean set estimators" which identify the set of near-minimizers of the empirical Fréchet functional and where the amount of "relaxation" vanishes as the number of data tends to infinity. On the theoretical side, our results include exact descriptions of which relaxation rates give weak consistency and which give strong consistency, as well as a description of an estimator which (assuming only the finiteness of certain moments and a mild condition on the metric entropy of the underlying metric space) adaptively finds the fastest possible relaxation rate for strongly consistent estimation. On the applied side, we consider the problem of estimating the set of Fermat-Weber points of an unknown distribution in the space of equidistant trees endowed with the tropical projective metric; in this setting, we provide an algorithm that provably implements our adaptive estimator, and we apply this method to real phylogenetic data.