This paper studies the convergence of mean field games with finite state space to mean field games with a continuous state space. We examine a space discretization of a diffusive dynamics, which is reminiscent of the Markov chain approximation method in stochasctic control, but also of finite difference numerical schemes; time remains continuous in the discretization, and the time horizon is arbitrarily long. We are mainly interested in the convergence of the solution of the associated master equations as the number of states tends to infinity. We present two approaches, to treat the case without or with common noise, both under monotonicity assumptions. The first one uses the system of characteristics of the master equation, which is the MFG system, to establish a convergence rate for the master equations without common noise and the associated optimal trajectories, both in case there is a smooth solution to the limit master equation and in case there is not. The second approach relies on the notion of monotone solutions introduced by Bertucci. In the presence of common noise, we show convergence of the master equations, with a convergence rate if the limit master equation is smooth, otherwise by compactness arguments.