A general class of mean field games are considered where the governing dynamics are controlled diffusions in $\mathbb{R}^d$. The optimization criterion is the long time average of a running cost function. Under various sets of hypotheses, we establish the existence of mean field game solutions. We also study the long time behavior of mean field game solutions associated with the finite horizon problem, and under the assumption of geometric ergodicity for the dynamics, we show that these converge to the ergodic mean field game solution as the horizon tends to infinity. Lastly, we study the associated $N$-player games, show existence of Nash equilibria, and establish the convergence of the solutions associated to Nash equilibria of the game to a mean field game solution as $N\to\infty$.