Preferential attachment networks with power law exponent $τ>3$ are known to exhibit a phase transition. There is a value $ρ_{\rm c}>0$ such that, for small edge densities $ρ\leq ρ_c$ every component of the graph comprises an asymptotically vanishing proportion of vertices, while for large edge densities $ρ>ρ_c$ there is a unique giant component comprising an asymptotically positive proportion of vertices. In this paper we study the decay in the size of the giant component as the critical edge density is approached from above. We show that the size decays very rapidly, like $\exp(-c/ \sqrt{ρ-ρ_c})$ for an explicit constant $c>0$ depending on the model implementation. This result is in contrast to the behaviour of the class of rank-one models of scale-free networks, including the configuration model, where the decay is polynomial. Our proofs rely on the local neighbourhood approximations of [Dereich, Morters, 2013] and recent progress in the theory of branching random walks [Gantert, Hu, Shi, 2011].