A Robust, Fully Adaptive M-estimator for Pointwise Estimation in Heteroscedastic Regression

We introduce a robust and fully adaptive method for pointwise estimation in heteroscedastic regression. We allow for noise and design distributions that are unknown and fulfill very weak assumptions only. In particular, we do not impose moment conditions on the noise distribution, and we allow for zero noise. Moreover, we do not require a strictly positive density for the design distribution. In a first step, we fix a bandwidth and construct M-estimators that consist of a contrast and a kernel. We then choose the contrast and the kernel that minimize the empirical variance and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive (Huber) minimax for contamination models. In a second step, we additionally choose a data-driven bandwidth via Lepski's method. This leads to an M-estimator that is adaptive with respect to the noise and design distributions and, additionally, adaptive with respect to the smoothness of an isotropic, locally polynomial target function. These results are also extended to anisotropic, locally constant target functions. Our data-driven approach provides, in particular, a level of robustness that adapts to the noise, contamination, and outliers. We finally conclude with a detailed discussion of our assumptions and an outlook on possible extensions.