Consider a channel ${\bf Y}={\bf X}+ {\bf N}$ where ${\bf X}$ is an $n$-dimensional random vector, and ${\bf N}$ is a Gaussian vector with a covariance matrix ${\bf \mathsf{K}}_{\bf N}$. The object under consideration in this paper is the conditional mean of ${\bf X}$ given ${\bf Y}={\bf y}$, that is ${\bf y} \to E[{\bf X}|{\bf Y}={\bf y}]$. Several identities in the literature connect $E[{\bf X}|{\bf Y}={\bf y}]$ to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean is derived. Specifically, for the Markov chain ${\bf U} \leftrightarrow {\bf X} \leftrightarrow {\bf Y}$, it is shown that the Jacobian of $E[{\bf U}|{\bf Y}={\bf y}]$ is given by ${\bf \mathsf{K}}_{\bf N}^{-1} {\bf Cov} ( {\bf X}, {\bf U} | {\bf Y}={\bf y})$. In the second part of the paper, via various choices of ${\bf U}$, the new identity is used to generalize many of the known identities and derive some new ones. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. Third, a new connection between the conditional cumulants and the conditional expectation is shown. In particular, it is shown that the $k$-th derivative of $E[X|Y=y]$ is the $(k+1)$-th conditional cumulant. The third part of the paper considers some applications. In a first application, the power series and the compositional inverse of $E[X|Y=y]$ are derived. In a second application, the distribution of the estimator error $(X-E[X|Y])$ is derived. In a third application, we construct consistent estimators (empirical Bayes estimators) of the conditional cumulants from an i.i.d. sequence $Y_1,...,Y_n$.