Regression modeling on stratified data: automatic and covariate-specific selection of the reference stratum with simple $L_1$-norm penalties

We consider regression models to be estimated on $K>1$ pre-defined strata of a sample. Denote by $\beta^*_{k,j}$ the theoretical parameter associated to the $j$-th covariate in the $k$-th stratum. It is common practice to first arbitrarily chose a reference stratum, say $\ell$, and perform inference based on the decomposition $\beta^*_{k,j} = \beta^*_{\ell,j} + \gamma^*_{k,j}$. In particular, $\ell_1$-penalized regression models can be constructed to recover non-zero parameters among the $\beta^*_{\ell,j}$'s and the $\gamma^*_{k,j}$'s. In this paper, we present a simple though efficient method that bypasses the arbitrary choice of the reference stratum at no cost. Its implementation can be done with available packages under a variety of models and, in the linear regression model, we show it is sparsistent under conditions similar to those ensuring sparsistency for an oracular version of the reference stratum strategy. Our empirical study further shows that our proposal performs at least as well as its competitors under the considered settings. As a final illustration, an analysis of road safety data is provided.