In Bayesian statistical inference and computationally intensive frequentist inference, one is interested in obtaining samples from a high dimensional, and possibly multi-modal target density. The challenge is to obtain samples from this target without any knowledge of the normalizing constant. Several approaches to this problem rely on Monte Carlo methods. One of the simplest such methods is the rejection sampler due to von Neumann. Here we introduce an auto-validating version of the rejection sampler via interval analysis. We show that our rejection sampler does provide us with independent samples from a large class of target densities in a guaranteed manner. We illustrate the efficiency of the sampler by theory and by examples in up to 10 dimensions. Our sampler is immune to the `pathologies' of some infamous densities including the witch's hat and can rigorously draw samples from piece-wise Euclidean spaces of small phylogenetic trees.