Approximate Bayesian Computation (ABC) is a popular computational method for likelihood-free Bayesian inference. The term "likelihood-free" refers to problems where the likelihood is intractable to compute or estimate directly, but where it is possible to generate simulated data $X$ relatively easily given a candidate set of parameters $θ$ simulated from a prior distribution. Parameters which generate simulated data within some tolerance $δ$ of the observed data $x^*$ are regarded as plausible, and a collection of such $θ$ is used to estimate the posterior distribution $θ\,|\,X\!=\!x^*$. Suitable choice of $δ$ is vital for ABC methods to return good approximations to $θ$ in reasonable computational time. While ABC methods are widely used in practice, particularly in population genetics, study of the mathematical properties of ABC estimators is still in its infancy. We prove that ABC estimates converge to the exact solution under very weak assumptions and, under slightly stronger assumptions, quantify the rate of this convergence. Our results can be used to guide the choice of the tolerance parameter $δ$.