We study the weak coupling limit of the Anderson Hamiltonian in the critical dimension $d=4$. In a perturbative sense, we prove Gaussian fluctuations about the Green's function of the Laplacian. The fluctuations are described by an explicit effective variance, up to a critical value of the coupling constant at which we expect a phase transition in the structure of the fluctuations. The proof is based on a combinatorial analysis of Feynman diagrams, and on a detailed study of the BPHZ renormalisation of the model. We characterise the limiting distribution in terms of primitive blow-ups, and prove that no Laplacian renormalisation is present. Our approach seems applicable to a broad class of equations.