Kinetic equations describe physical processes in a high-dimensional phase space and are often simulated using Markov process-based Monte Carlo routines. The quantities of interest are typically defined on the lower-dimensional position space and estimated on a grid (histogram). In several applications, such as the construction of diffusion Monte Carlo-like techniques and variance prediction for particle tracing Monte Carlo methods, the cell escape probabilities, i.e., the probabilities with which particles escape a grid cell during one step of the Markov process, are of interest. In this paper, we derive formulas to calculate the cell escape probabilities for common mesh elements in one, two, and three dimensions. Deterministic calculation of cell escape probabilities in higher dimensions becomes expensive and prone to quadrature errors due to the involved high-dimensional integrals. We therefore also introduce a stochastic Monte Carlo algorithm to calculate the escape probabilities, which is more robust at the cost of a statistical error. The code used to perform the numerical experiments and accompanying GeoGebra tutorials are openly available at https://gitlab.kuleuven.be/numa/public/escape-probabilities-markov-processes.