Let $X$ be a random walk on the torus of side length $N$ in dimension $d\geq 3$ with uniform starting point, and $t_{\text{cov}}$ be the expected value of its cover time, which is the first time that $X$ has visited every vertex of the torus at least once. For $α> 0$, the set $\mathcal{L}^α$ of $α$-late points consists of those points not visited by $X$ at time $αt_{\text{cov}}$. We prove the existence of a value $α_* \in (\frac12,1)$ across which $\mathcal{L}^α$ trivialises as follows: for all $α> α_*$ and $ε\geq N^{-c}$ there exists a coupling of $\mathcal{L}^α$ and two occupation sets $\mathcal{B}^{α_\pm}$ of i.i.d. Bernoulli fields having the same density as $\mathcal{L}^{α\pm ε}$, which is asymptotic to $N^{-(α\pmε)d}$, with the property that the inclusion $ \mathcal{B}^{α_+} \subseteq \mathcal{L}^α \subseteq \mathcal{B}^{α_-}$ holds with high probability as $N \to \infty$. On the contrary, when $α\leq α_*$ there is no such coupling. Corresponding results also hold for the vacant set of random interlacements at high intensities. The transition at $α_*$ corresponds to the (dis-)appearance of `double-points' (i.e. neighboring pairs of points) in $\mathcal{L}^α$. We further describe the law of $\mathcal{L}^α$ for $α>\frac12$ by adding independent patterns to $\mathcal{B}^{α_{\pm}}$. In dimensions $d \geq 4$ these are exactly all two-point sets. When $d=3$ one must also include all connected three-point sets, but no other.