Under ideal conditions, the probability density function (PDF) of a random variable, such as a sensor measurement, would be well known and amenable to computation and communication tasks. However, this is often not the case, so the user looks for some other PDF that approximates the true but intractable PDF. Conservativeness is a commonly sought property of this approximating PDF, especially in distributed or unstructured data systems where the data being fused may contain un-known correlations. Roughly, a conservative approximation is one that overestimates the uncertainty of a system. While prior work has introduced some definitions of conservativeness, these definitions either apply only to normal distributions or violate some of the intuitive appeal of (Gaussian) conservative definitions. This work provides a general and intuitive definition of conservativeness that is applicable to any probability distribution that is a measure over $\mathbb{R}^m$ or an infinite subset thereof, including multi-modal and uniform distributions. Unfortunately, we show that this strong definition of conservative cannot be used to evaluate data fusion techniques. Therefore, we also describe a weaker definition of conservative and show it is preserved through common data fusion methods such as the linear and log-linear opinion pool, and homogeneous functionals, assuming the input distributions can be factored into independent and common distributions. In addition, we show that after fusion, weak conservativeness is preserved by Bayesian updates. These strong and weak definitions of conservativeness can help design and evaluate potential correlation-agnostic data fusion techniques.