In topological data analysis, persistent homology characterizes robust topological features in data and it has a summary representation, called a persistence diagram. Statistical research for persistence diagrams have been actively developed, and the persistence weighted kernel shows several advantages over other statistical methods for persistence diagrams. If data is drawn from some probability distribution, the corresponding persistence diagram have randomness. Then, the expectation of the persistence diagram by the persistence weighted kernel is well-defined. In this paper, we study relationships between a probability distribution and the persistence weighted kernel in the viewpoint of (1) the strong law of large numbers and the central limit theorem, (2) a confidence interval to estimate the expectation of the persistence weighted kernel numerically, and (3) the stability theorem to ensure the continuity of the map from a probability distribution to the expectation. In numerical experiments, we demonstrate our method gives an interesting counterexample to a common view in topological data analysis.