In [14], Nourdin and Peccati combined the Malliavin calculus and Stein's method of normal approximation to associate a rate of convergence to the celebrated fourth moment theorem [19] of Nualart and Peccati. Their analysis, known as the Malliavin-Stein method nowadays, has found many applications towards stochastic geometry, statistical physics and zeros of random polynomials, to name a few. In this article, we further explore the relation between these two fields of mathematics. In particular, we construct exchangeable pairs of Brownian motions and we discover a natural link between Malliavin operators and these exchangeable pairs. By combining our findings with E. Meckes' infinitesimal version of exchangeable pairs, we can give another proof of the quantitative fourth moment theorem. Finally, we extend our result to the multidimensional case.