The aim of the paper is to address the long time behavior of the Kuramoto model of mean-field coupled phase rotators, subject to white noise and quenched frequencies. We analyse the influence of the fluctuations of both thermal noise and frequencies (seen as a disorder) on a large but finite population of $N$ rotators, in the case where the law of the disorder is symmetric. On a finite time scale $[0, T]$, the system is known to be self-averaging: the empirical measure of the system converges as $N\to \infty$ to the deterministic solution of a nonlinear Fokker-Planck equation which exhibits a stable manifold of synchronized stationary profiles for large interaction. On longer time scales, competition between the finite-size effects of the noise and disorder makes the system deviate from this mean-field behavior. In the main result of the paper we show that on a time scale of order $ \sqrt{N}$ the fluctuations of the disorder prevail over the fluctuations of the noise: we establish the existence of disorder-induced traveling waves for the empirical measure along the stationary manifold. This result is proved for fixed realizations of the disorder and emphasis is put on the influence of the asymmetry of these quenched frequencies on the direction and speed of rotation of the system. Asymptotics on the drift are provided in the limit of small disorder.