Stochastic inverse problems considered in this article consist of estimating the probability distributions of intrinsically random inputs of computer models. These estimations are based on observable outputs affected by model noise, and such problems are increasingly examined in parametric Bayesian contexts where the parameters of the targeted input distributions are affected by epistemic uncertainties. With the aim of improving the meaningfulness of solutions found by statistical algorithms -- in the sense that forward simulations based on such solutions must lead to relevant observables -- we derive new prior constraints using the principles of global sensitivity analysis and information theory. Primarily formalized as constraints on covariances in Gaussian linear or linearizable situations, they reflect the idea that the solution should explain most of the observable uncertainty, while the model noise remains a secondary factor of this uncertainty. Simulated experiments highlight that, when injected into stochastic inversion algorithms, these constraints can indeed limit the influence of model noise on the result. They provide hope for future extensions in more general frameworks, for example through the use of linear Gaussian mixtures.