We consider the Schramm-Loewner evolution (SLE$_κ$) with $κ=4$, the critical value of $κ> 0$ at or below which SLE$_κ$ is a simple curve and above which it is self-intersecting. We show that the range of an SLE$_4$ curve is a.s. conformally removable, answering a question posed by Sheffield. Such curves arise as the conformal welding of a pair of independent critical ($γ=2$) Liouville quantum gravity (LQG) surfaces along their boundaries and our result implies that this conformal welding is unique. In order to establish this result, we give a new sufficient condition for a set $X \subseteq {\mathbf C}$ to be conformally removable which applies in the case that $X$ is not necessarily the boundary of a simply connected domain.