We study mean field games and corresponding $N$-player games in continuous time over a finite time horizon where the position of each agent belongs to a finite state space. As opposed to previous works on finite state mean field games, we use a probabilistic representation of the system dynamics in terms of stochastic differential equations driven by Poisson random measures. Under mild assumptions, we prove existence of solutions to the mean field game in relaxed open-loop as well as relaxed feedback controls. Relying on the probabilistic representation and a coupling argument, we show that mean field game solutions provide symmetric $ε_N$-Nash equilibria for the $N$-player game, both in open-loop and in feedback strategies (not relaxed), with $ε_N\leq \frac{\text{constant}}{\sqrt{N}}$. Under stronger assumptions, we also find solutions of the mean field game in ordinary feedback controls and prove uniqueness either in case of a small time horizon or under monotonicity.