In this work, we study the convergence rate of the $N$-player LQG game with a Markov chain common noise towards its asymptotic Mean Field Game. By postulating a Markovian structure via two auxiliary processes for the first and second moments of the Mean Field Game equilibrium and applying the fixed point condition in Mean Field Game, we first provide the characterization of the equilibrium measure in Mean Field Game with a finite-dimensional Riccati system of ODEs. Additionally, with an explicit coupling of the optimal trajectory of the $N$-player game driven by $N$ dimensional Brownian motion and Mean Field Game counterpart driven by one-dimensional Brownian motion, we obtain the convergence rate $O(N^{-1/2})$ with respect to 2-Wasserstein distance.