The classical unbiasedness condition utilized e.g. by the best linear unbiased estimator (BLUE) is very stringent. By softening the "global" unbiasedness condition and introducing component-wise conditional unbiasedness conditions instead, the number of constraints limiting the estimator's performance can in many cases significantly be reduced. In this work we investigate the component-wise conditionally unbiased linear minimum mean square error (CWCU LMMSE) estimator for different model assumptions. The prerequisites in general differ from the ones of the LMMSE estimator. We first derive the CWCU LMMSE estimator under the jointly Gaussian assumption of the measurements and the parameters. Then we focus on the linear model and discuss the CWCU LMMSE estimator for jointly Gaussian parameters, and for mutually independent (and otherwise arbitrarily distributed) parameters, respectively. In all these cases the CWCU LMMSE estimator incorporates the prior mean and the prior covariance matrix of the parameter vector. For the remaining cases optimum linear CWCU estimators exist, but they may correspond to globally unbiased estimators that do not make use of prior statistical knowledge about the parameters. Finally, the beneficial properties of the CWCU LMMSE estimator are demonstrated with the help of a well-known channel estimation application.