Some Large Sample Results for the Method of Regularized Estimators

We present a general framework for studying regularized estimators; i.e., estimation problems wherein "plug-in" type estimators are either ill-defined or ill-behaved. We derive primitive conditions that imply consistency and asymptotic linear representation for regularized estimators, allowing for slower than $\sqrt{n}$ estimators as well as infinite dimensional parameters. We also provide data-driven methods for choosing tuning parameters that, under some conditions, achieve the aforementioned results. We illustrate the scope of our approach by studying a wide range of applications, revisiting known results and deriving new ones.