Given a set of moment restrictions (MRs) that overidentify a parameter $θ$, we investigate a semiparametric Bayesian approach for inference on $θ$ that does not restrict the data distribution $F$ apart from the MRs. As main contribution, we construct a degenerate Gaussian process prior that, conditionally on $θ$, restricts the $F$ generated by this prior to satisfy the MRs with probability one. Our prior works even in the more involved case where the number of MRs is larger than the dimension of $θ$. We demonstrate that the corresponding posterior for $θ$ is computationally convenient. Moreover, we show that there exists a link between our procedure, the Generalized Empirical Likelihood with quadratic criterion and the limited information likelihood-based procedures. We provide a frequentist validation of our procedure by showing consistency and asymptotic normality of the posterior distribution of $θ$. The finite sample properties of our method are illustrated through Monte Carlo experiments and we provide an application to demand estimation in the airline market.