The class of stochastic matrices that have a stochastic $c$-th root for infinitely many natural numbers $c$ is introduced and studied. Such matrices are called arbitrarily finely divisible, and generalise the class of infinitely divisible matrices. In particular, if $A$ is a transition matrix for a Markov process over some time period, then arbitrarily finely divisibility of $A$ is the necessary and sufficient condition for the existence of transition matrices corresponding to this Markov process over arbitrarily short periods. In this paper, we lay the foundation for research into arbitrarily finely divisible matrices and demonstrate the concepts using specific examples of $2 \times 2$ matrices, $3 \times 3$ circulant matrices, and rank-two matrices.