We construct a continuous family of exchangeable pairs by perturbing the random variable through diffusion processes on manifold in order to apply Stein method to certain geometric settings. We compare our perturbation by diffusion method with other approaches of building exchangeable pairs and show that our perturbation scheme cooperates with the infinitesimal version of Stein's method harmoniously. More precisely, our exchangeable pairs satisfy a key condition in the infinitesimal Stein's method in general. Based on the exchangeable pairs, we are able to extend the approximate normality of eigenfunctions of Laplacian on compact manifold to eigenfunctions of Witten Laplacian, which is of the form:$Δ_w = Δ- \nabla H$. We then apply our abstract theorem to recover a central limit result of linear statistics on sphere. Finally, we prove an an infinitesimal version of Stein's method for exponential distribution and combine it with our continuous family of exchangeable pairs to extend an approximate exponentiality result of $|Tr U|^2$, where $Tr U$ is the trace of the first power of a matrix $U$ sampled from the Haar measure of unitary group, to arbitrary power and its analog for general circular ensemble.