In this paper, we propose a new efficient algorithm to compute the value function for zero-sum stopping games featuring two players with opposing interests. This can be seen as a game version of the ''forward algorithm'' for (one-player) optimal stopping problem, first introduced by Irle [6] for discrete-time Markov processes and later revisited by Miclo \& Villeneuve [8] for continuous-time Markov processes on general state spaces. This paper focuses on a game driven by a homogeneous Markov process taking values in a finite state space and also discusses about the number of iterations needed. Illustrated computational implementations for a few particular examples are also provided.