We give a probabilistic characterization of the set of measures that can be represented by the matrix product ansatz. By suitably enlarging the state space, we show that a probability measure can be described in terms of non negative matrices by the {\it Matrix Product Ansatz}, if and only if it can be written as a mixture of inhomogeneous product measures where the mixing law is a Markov bridge. We give a constructive procedure to identify such probabilistic features. We illustrate the result by examples and show that existing probabilistic representations of the invariant measures of non equilibrium interacting particle systems can be obtained from the matrix product ansatz by this general procedure.