In $M$-estimation under standard asymptotics, the weak convergence combined with the polynomial type large deviation estimate of the associated statistical random field Yoshida (2011) provides us with not only the asymptotic distribution of the associated $M$-estimator but also the convergence of its moments, the latter playing an important role in theoretical statistics. In this paper, we study the above program for statistical random fields of multiple and also possibly mixed-rates type in the sense of Radchenko (2008) where the associated statistical random fields may be non-differentiable and may fail to be locally asymptotically quadratic. Consequently, a very strong mode of convergence of a wide range of regularized $M$-estimators is ensured. The results are applied to regularized estimation of an ergodic diffusion observed at high frequency.