We consider the $[0,1]$-valued solution $(u_{t,x}:t\geq 0, x\in \mathbb R)$ to the one dimensional stochastic reaction diffusion equation with Wright-Fisher noise \[\partial_t u= \partial_x^2 u + f(u) + ε\sqrt{u(1-u)} \dot W.\] Here, $W$ is a space-time white noise, $ε> 0$ is the noise strength, and $f$ is a continuous function on $[0,1]$ satisfying $\sup_{z\in [0,1]}|f(z)|/ \sqrt{z(1-z)} < \infty.$ We assume the initial data satisfies $1 - u_{0,-x} = u_{0,x} = 0$ for $x$ large enough. Recently, it was proved in (Comm. Math. Phys. \textbf{384} (2021), no. 2) that the front of $u_t$ propagates with a finite deterministic speed $V_{f,ε}$, and under slightly stronger conditions on $f$, the asymptotic behavior of $V_{f,ε}$ was derived as the noise strength $ε$ approaches $\infty$. In this paper we complement the above result by obtaining the asymptotic behavior of $V_{f,ε}$ as the noise strength $ε$ approaches $0$: for a given $p\in [1/2,1)$, if $f(z)$ is non-negative and is comparable to $z^p$ for sufficiently small $z$, then $V_{f,ε}$ is comparable to $ε^{-2\frac{1-p}{1+p}}$ for sufficiently small $ε$.