Analysis of experimental data must sometimes deal with abrupt changes in the distribution of measured values. Setting upper limits on signals usually involves a veto procedure that excludes data not described by an assumed statistical model. We show how to implement statistical estimates of physical quantities (such as upper limits) that are valid without assuming a particular family of statistical distributions, while still providing close to optimal values when the data is from an expected distribution (such as Gaussian or exponential). This new technique can compute statistically sound results in the presence of severe non-Gaussian noise, relaxes assumptions on distribution stationarity and is especially useful in automated analysis of large datasets, where computational speed is important.