In this work the issue of Bayesian inference for stationary data is addressed. Therefor a parametrization of a statistically suitable subspace of the the shift-ergodic probability measures on a Cartesian product of some finite state space is given using an inverse limit construction. Moreover, an explicit model for the prior is given by taking into account an additional step in the usual stepwise sampling scheme of data. An update to the posterior is defined by exploiting this augmented sample scheme. Thereby, its model-step is updated using a measurement of the empirical distances between the model classes.