We consider the space of probability measures on a discrete set $X$, endowed with a dynamical optimal transport metric. Given two probability measures supported in a subset $Y \subseteq X$, it is natural to ask whether they can be connected by a constant speed geodesic with support in $Y$ at all times. Our main result answers this question affirmatively, under a suitable geometric condition on $Y$ introduced in this paper. The proof relies on an extension result for subsolutions to discrete Hamilton-Jacobi equations, which is of independent interest.