Many inverse problems require reconstructing physical fields from limited and noisy data while incorporating known governing equations. A growing body of work within probabilistic numerics formalizes such tasks via Bayesian inference in function spaces by assigning a physically meaningful prior to the latent field. In this work, we demonstrate that Brownian bridge Gaussian processes can be viewed as a softly-enforced physics-constrained prior for the Poisson equation. We first show equivalence between the variational problem associated with the Poisson equation and a kernel ridge regression objective. Then, through the connection between Gaussian process regression and kernel methods, we identify a Gaussian process for which the posterior mean function and the minimizer to the variational problem agree, thereby placing this PDE-based regularization within a fully Bayesian framework. This connection allows us to probe different theoretical questions, such as convergence and behavior of inverse problems. We then develop a finite-dimensional representation in function space and prove convergence of the projected prior and resulting posterior in Wasserstein distance. Finally, we connect the method to the important problem of identifying model-form error in applications, providing a diagnostic for model misspecification.