The circular unitary ensemble and its generalizations concern a random matrix from a compact classical group $\mathrm{U}(N)$, $\mathrm{SU}(N)$, $\mathrm{O}(N)$, $\mathrm{SO}(N)$ or $\mathrm{USp}(N)$ distributed according to the Haar measure. The eigenvalues are known to be very evenly distributed on the unit circle. In this paper, we study the distance from the empirical measure of the eigenvalues to uniformity in the quadratic Wasserstein metric $W_2$. After finding the exact value of the expected value and the variance, we deduce a limit law for the square of the Wasserstein distance. We reformulate our results in terms of the $L^2$ average of the number of eigenvalues in circular arcs, and also in terms of the characteristic polynomial of the matrix on the unit circle.