Let $R_n$ be the range of a critical branching random walk with $n$ particles on $\mathbb Z^d$, which is the set of sites visited by a random walk indexed by a critical Galton--Watson tree conditioned on having exactly $n$ vertices. For $d\in\{3, 4, 5\}$, we prove that $n^{-\frac{d-2}4} \mathtt{cap}^{(d)}(R_n)$, the renormalized capacity of $R_n$, converges in law to the capacity of the support of the integrated super-Brownian excursion. The proof relies on a study of the intersection probabilities between the critical branching random walk and an independent simple random walk on $\mathbb Z^d$.