We prove rigorously that the ferromagnetic Ising model on any nonamenable Cayley graph undergoes a continuous (second-order) phase transition in the sense that there is a unique Gibbs measure at the critical temperature. The proof of this theorem is quantitative and also yields power-law bounds on the magnetization at and near criticality. Indeed, we prove more generally that the magnetization $\langle σ_o \rangle_{β,h}^+$ is a locally Hölder-continuous function of the inverse temperature $β$ and external field $h$ throughout the non-negative quadrant $(β,h)\in [0,\infty)^2$. As a second application of the methods we develop, we also prove that the free energy of Bernoulli percolation is twice differentiable at $p_c$ on any transitive nonamenable graph.