We consider a non-linear heat equation $\partial_t u = Δu + B(u,Du)+P(u)$ posed on the $d$-dimensional torus, where $P$ is a polynomial of degree at most $3$ and $B$ is a bilinear map that is not a total derivative. We show that, if the initial condition $u_0$ is taken from a sequence of smooth Gaussian fields with a specified covariance, then $u$ exhibits norm inflation with high probability. A consequence of this result is that there exists no Banach space of distributions which carries the Gaussian free field on the 3D torus and to which the DeTurck-Yang-Mills heat flow extends continuously, which complements recent well-posedness results in arXiv:2111.10652 and arXiv:2201.03487. Another consequence is that the (deterministic) non-linear heat equation exhibits norm inflation, and is thus locally ill-posed, at every point in the Besov space $B^{-1/2}_{\infty,\infty}$; the space $B^{-1/2}_{\infty,\infty}$ is an endpoint since the equation is locally well-posed for $B^η_{\infty,\infty}$ for every $η>-\frac12$.