A random set of points in Euclidean space is called `rigid' or `hyperuniform' if the number of points falling inside any given region has significantly smaller fluctuations than the corresponding number for a set of i.i.d. random points. This phenomenon has received considerable attention in recent years, due to its appearance in random matrix theory, the theory of Coulomb gases and zeros of random analytic functions. However, most of the published results are in dimensions one and two. This paper gives the first proof of hyperuniformity in a Coulomb type system in dimension three, known as the hierarchical Coulomb gas. This is a simplified version of the actual 3D Coulomb gas. The interaction potential in this model, inspired by Dyson's hierarchical model of the Ising ferromagnet, has a hierarchical structure and is locally an approximation of the Coulomb potential. Hyperuniformity is proved at both macroscopic and microscopic scales, with upper and lower bounds for the order of fluctuations that match up to logarithmic factors. The fluctuations have cube-root behavior, in agreement with a well-known prediction for the 3D Coulomb gas. For completeness, analogous results are also proved for the 2D hierarchical Coulomb gas and the 1D hierarchical log gas.