Weak convergence of the regularization path in penalized M-estimation

We consider an estimator $\hbbeta_n(\param)$ defined as the element $\bphi\in\bPhi$ minimizing a contrast process $\pencontrast(\bphi,\param)$ for each $\param$. We give some general results for deriving the weak convergence of $\sqrt{n}(\hbbeta_n-\bbeta)$ in the space of bounded functions, where, for each $\param$, $\bbeta(\param)$ is the $\bphi\in\bPhi$ minimizing the limit of $\pencontrast(\bphi,\param)$ as $n\to\infty$. These results are applied in the context of penalized M-estimation, that is, when $\pencontrast(\bphi,\param)=M_n(\bphi)+\param J_n(\bphi)$, where $M_n$ is a usual contrast process and $J_n$ a penalty such as the $\ell^1$ norm or the squared $\ell^2$ norm. The function $\hbbeta_n$ is then called a \emph{regularization path}. For instance we show that the central limit theorem established for the lasso estimator in Knight and Fu (2000) continues to hold in a functional sense for the regularization path. Other examples include various possible contrast processes for $M_n$ such as those considered in Pollard (1985). To illustrate these results in the lasso case, we propose a test statistic based on the regularization path whose asymptotic distribution is known under the null hypothesis $H_0:\bbeta=0$. The performance of the test is assessed on synthetic data.