We construct the canonical Brownian motion on the gasket of conformal loop ensembles (CLE$_κ$) for $κ\in (4,8)$ (which is the range of parameter values in which loops of the CLE$_κ$ can intersect themselves, each other, and the domain boundary). More precisely, we show that there is a unique diffusion process on the CLE$_κ$ gasket whose law depends locally on the CLE$_κ$ and satisfies certain natural properties such as translation-invariance and scale-invariance (modulo time change). We characterize the diffusion process by its resistance form and show in particular that there is a unique resistance form on the CLE$_κ$ gasket that is locally determined by the CLE$_κ$ and satisfies certain natural properties such as translation-invariance and scale-covariance. We conjecture that the CLE$_κ$ Brownian motion describes the scaling limit of simple random walk on statistical mechanics models in two dimensions that converge to CLE$_κ$. In future work the results of this paper will be used to show that this is the case with $κ=6$ for critical percolation on the triangular lattice.