Concentration of quadratic forms and aggregation of affine estimators

This paper deals with aggregation of estimators in the context of regression with fixed design, with heteroscedastic and subgaussian noise. We relate the task of aggregating a finite family of affine estimators to the concentration of quadratic forms of the noise vector, and we derive sharp oracle inequalities in deviation for model selection type aggregation of affine estimators when the noise is subgaussian. Explicit numerical constants are given for Gaussian noise. Then we present a new concentration result that is sharper than the Hanson-Wright inequality under the Bernstein condition on the noise. This allows us to improve the sharp oracle inequality obtained in the subgaussian case. Finally, we show that up to numerical constants, the optimal sparsity oracle inequality previously obtained for Gaussian noise holds in the subgaussian case. The exact knowledge of the variance of the noise is not needed to construct the estimator that satisfies the sparsity oracle inequality.