On the trace of a discrete-time simple random walk on $\mathbb{Z}^d$ for $d\geq 2$, we consider the evolution of favorite sites, i.e., sites that achieve the maximal local time at a certain time. For $d=2$, we show that almost surely three favorite sites occur simultaneously infinitely often and eventually there is no simultaneous occurrence of four favorite sites. For $d\geq 3$, we derive sharp asymptotics of the number of favorite sites. This answers an open question of Erdős and Révész (1987), which was brought up again in Dembo (2005).