This article studies the expected occupancy probabilities on an alphabet. Unlike the standard situation, where observations are assumed to be independent and identically distributed (iid), we assume that they follow a regime switching Markov chain. For this model, we 1) give finite sample bounds on the occupancy probabilities, and 2) provide detailed asymptotics in the case where the underlying distribution is regularly varying. We find that, in the regularly varying case, the finite sample bounds are rate optimal and have, up to a constant, the same rate of decay as the asymptotic result.