Let $M$ be a $d$-dimensional connected compact Riemannian manifold with boundary $\partial M$, let $V\in C^2(M)$ such that $μ({\rm d} x):={\rm e}^{V(x)}{\rm d} x$ is a probability measure, and let $X_t$ be the diffusion process generated by $L:=Δ+\nabla V$ with $τ:=\inf\{t\ge 0: X_t\in\partial M\}$. Consider the empirical measure $μ_t:=\frac 1 t \int_0^t δ_{X_s}{\rm d} s$ under the condition $t<τ$ for the diffusion process. If $d\le 3$, then for any initial distribution not fully supported on $\partial M$, \begin{align*} &c\sum_{m=1}^\infty \frac{2}{(λ_m-λ_0)^2} \le \liminf_{t\to \infty} \inf_{T\ge t} \Big\{t {\mathbb E}\big[\mathbb W_2(μ_t, μ_0)^2\big|T<τ\big]\Big\} \\ &\le \limsup_{t\to \infty} \sup_{T\ge t} \Big\{ t \mathbb E\big[\mathbb W_2(μ_t, μ_0)^2\big|T<τ\big] \Big\}\le \sum_{m=1}^\infty \frac{2}{(λ_m-λ_0)^2}\end{align*} holds for some constant $c\in (0,1]$ with $c=1$ when $\partial M$ is convex, where $μ_0:= φ_0^2μ$ for the first Dirichet eigenfunction $φ_0$ of $L$, $\{λ_m\}_{m\ge 0}$ are the Dirichlet eigenvalues of $-L$ listed in the increasing order counting multiplicities, and the upper bound is finite if and only if $d\le 3$. When $d=4$, $\sup_{T\ge t} \mathbb E\big[\mathbb W_2(μ_t, μ_0)^2\big|T<τ\big] $ decays in the order $t^{-1}\log t$, while for $d\ge 5$ it behaves like $t^{-\frac 2 {d-2}}$, as $t\to\infty$.