In the application of Bayesian methods to metrology, pre-data probabilities play a critical role in the estimation of the model uncertainty. Following the observation that distributions form Riemann's manifolds, methods of differential geometry can be applied to ensure covariant priors and uncertainties independent of parameterization. Paradoxes were found in multi-parameter problems and alternatives were developed; but, when different parameters are of interest, covariance may be lost. This paper overviews information geometry, investigates some key paradoxes, and proposes solutions that preserve covariance.