We investigate Ramsey properties of a random graph model in which random edges are added to a given dense graph. Specifically, we determine lower and upper bounds on the function $p=p(n)$ that ensures that for any dense graph $G_n$ a.a.s. every 2-colouring of the edges of $G_n\cup G(n,p)$ admits a monochromatic copy of the complete graph $K_r$. These bounds are asymptotically sharp for the cases when $r\geq 5$ is odd and almost sharp when $r\geq 4$ is even. Our proofs utilise recent results on the threshold for asymmetric Ramsey properties in $G(n,p)$ and the method of dependent random choice.