We describe Bayesian inference for the parameters of Gaussian models of bounded data protected by differential privacy. Using this setting, we demonstrate that analysts can and should take constraints imposed by the bounds into account when specifying prior distributions. Additionally, we provide theoretical and empirical results regarding what classes of default priors produce valid inference for a differentially private release in settings where substantial prior information is not available. We discuss how these results can be applied to Bayesian inference for regression with differentially private data.