We consider the recovery of an unknown function $f$ from a noisy observation of the solution $u_f$ to a partial differential equation that can be written in the form $\mathcal{L} u_f=c(f,u_f)$, for a differential operator $\mathcal{L}$ that is rich enough to recover $f$ from $\mathcal{L} u_f$. Examples include the time-independent Schrödinger equation $Δu_f = 2u_ff$, the heat equation with absorption term $(\partial_t -Δ_x/2) u_f=fu_f$, and the Darcy problem $\nabla\cdot (f \nabla u_f) = h$. We transform this problem into the linear inverse problem of recovering $\mathcal{L} u_f$ under the Dirichlet boundary condition, and show that Bayesian methods with priors placed either on $u_f$ or $\mathcal{L} u_f$ for this problem yield optimal recovery rates not only for $u_f$, but also for $f$. We also derive frequentist coverage guarantees for the corresponding Bayesian credible sets. Adaptive priors are shown to yield adaptive contraction rates for $f$, thus eliminating the need to know the smoothness of this function. The results are illustrated by numerical experiments on synthetic data sets.