We consider the problem of distributed estimation under the Bayesian criterion and explore the design of optimal quantizers in such a system. We show that, for a conditionally unbiased and efficient estimator at the fusion center and when local observations have identical distributions, it is optimal to partition the local sensors into groups, with all sensors within a group using the same quantization rule. When all the sensors use identical number of decision regions, use of identical quantizers at the sensors is optimal. When the network is constrained by the capacity of the wireless multiple access channel over which the sensors transmit their quantized observations, we show that binary quantizers at the local sensors are optimal under certain conditions. Based on these observations, we address the location parameter estimation problem and present our optimal quantizer design approach. We also derive the performance limit for distributed location parameter estimation under the Bayesian criterion and find the conditions when the widely used threshold quantizer achieves this limit. We corroborate this result using simulations. We then relax the assumption of conditionally independent observations and derive the optimality conditions of quantizers for conditionally dependent observations. Using counter-examples, we also show that the previous results do not hold in this setting of dependent observations and, therefore, identical quantizers are not optimal.