Pivotal Estimation of Nonparametric Functions via Square-root Lasso

We propose a self-tuning square-root lasso method that simultaneously resolves three important practical problems in high-dimensional regression analysis, namely it handles the unknown scale, heteroscedasticity, and (drastic) non-Gaussianity of the noise. In addition our analysis allows for badly behaved designs, e.g. perfectly collinear regressors, and generates sharp bounds on performance even in extreme cases, such as the infinite variance case and the noiseless case, in contrast to lasso. We systematically establish various non-asymptotic bounds for square-root lasso performance including prediction norm rate, L-1-rate, L-inf-rate, and sharp sparsity bound. In order to cover heteroskedastic non-Gaussian noise, we rely on moderate deviation theory for selfnormalized sums to achieve Gaussian-like results under weak conditions. Moreover, we derive bounds on the performance of ordinary least square (ols) applied to the model selected by square-root lasso accounting for possible misspecification of the selected model. Under mild conditions the rate of convergence of ols post square-root lasso is no worse than square-root lasso even with a misspecified selected model and possibly better otherwise.