Moment convergence in regularized estimation under multiple and mixed-rates asymptotics

In $M$-estimation under standard asymptotics, the weak convergence combined with a large deviation estimate of the associated statistical random field provides us with a general tool for deriving not only the asymptotic distribution of the associated $M$-estimator but also the convergence of its moments, where the latter plays an important role in theoretical statistics [22]. However, the general tools cannot directly apply in several situations including regularized sparse $M$-estimation. In this paper, we study the above program for statistical random fields of multiple and mixed-rates type. Specifically, we will provide a general machinery to deduce polynomial type uniform tail-probability estimate of a general scaled $M$-estimator under mixed-rates asymptotics in the sense of [12], where the associated statistical random fields may be non-differentiable and may fail to be locally asymptotically quadratic. As a result, our result enables us to deduce convergence of moments of a wide range of regularized, possibly sparse $M$-estimators.