Plug-in Regularized Estimation of High-Dimensional Parameters in Nonlinear Semiparametric Models

We propose an l1-regularized M-estimator for a high-dimensional sparse parameter that is identified by a class of semiparametric conditional moment restrictions (CMR). We estimate the nonparametric nuisance parameter by modern machine learning methods. Plugging the first-stage estimate into the CMR, we construct the M-estimator loss function for the target parameter so that its gradient is insensitive (formally, Neyman-orthogonal) with respect to the first-stage regularization bias. As a result, the estimator achieves oracle convergence rate \sqrt{k \log p/n}, where oracle knows the true first stage and solves only a parametric problem. We apply our results to conditional moment models with missing data, games of incomplete information and treatment effects in regression models with non-linear link functions.