A randomly perturbed graph $G^p = G_α\cup G(n,p)$ is obtained by taking a deterministic $n$-vertex graph $G_α= (V, E)$ with minimum degree $δ(G)\geq αn$ and adding the edges of the binomial random graph $G(n,p)$ defined on the same vertex set $V$. For which value $p$ (depending on $α$) does the graph $G^p$ contain a $K_r$-factor (a spanning collection of vertex-disjoint $K_r$-copies) with high probability? The order of magnitude of the minimal value of $p$ has been determined whenever $α\neq 1- \frac{s}{r}$ for an integer $s$ (see Han, Morris, and Treglown [RSA, 2021] and Balogh, Treglown, and Wagner [CPC, 2019]). We establish the minimal probability $p_s$ (up to a constant factor) for all values of $α= 1-\frac{s}{r} \leq \frac 12$, and show that the threshold exhibits a polynomial jump at $α= 1-\frac{s}{r}$ compared to the surrounding intervals. An extremal example $G_α$ which shows that $p_s$ is optimal up to a constant factor differs from the previous (usually multipartite) examples in containing a pseudorandom induced subgraph.