Weingarten functions provide a tool for computing Haar measure matrix integrals of polynomials in the matrix entries. An important property of Weingarten functions, is their particularly simple large $N$ limits. In 2017 Benoit Collins and Sho Matsumoto studied when this limit holds for Weingarten functions associated to integrals of products of $2n$ matrix entries, as $n \to \infty$, together with the matrix size $N$. They showed that the large $N$ limit is uniformly achieved as long as $n=o(N^{4/7})$, a result which already has applications to strong asymptotic freeness. However, their result is not optimal. They conjectured that their result should actually hold up to $n=o(N^{2/3})$ which is optimal. We prove this conjecture for the matrix groups $G \in \{\mathrm{U}(N)$, $\mathrm{O}(N)$, $\mathrm{Sp}(N)\}$. The proof proceeds by introducing a Markov process on permutations (pairings) which we call the unitary (orthogonal) $\textit{Weingarten process}$. We believe this process may have further applications to the theory of Weingarten functions. We also prove two new bounds regarding the large $N$ limit of the Weingarten function in the regimes when $n=o(N^{4/5})$, and $n=o(N)$.