Markov kernels play a decisive role in probability and mathematical statistics theories, and are an extension of the concepts of sigma-field and statistic. Concepts such as independence, sufficiency, completeness, ancillarity or conditional distribution have been extended previously to Markov kernels. In this paper, the concept of conditional expectation of a Markov kernel given another is introduced, setting its first properties. An application to clinical diagnosis is provided, obtaining {an} optimality property of the predictive values of a diagnosis test. In a statistical framework, this new probabilistic tool is used to extend to Markov kernels the theorems of {Rao-Blackwell} and Lehmann-Scheffé. A result about the completeness of a sufficient statistic is obtained in passing by properly enlarging the family of probabilities. As a final statistical scholium, a generalization of a result about the completeness of the family of nonrandomized estimators is given.