On the efficiency of the de-biased Lasso

We consider the high-dimensional linear regression model $Y = X \beta^0 + \epsilon$ with Gaussian noise $\epsilon$ and Gaussian random design $X$. We assume that $\Sigma_0:= {{\rm I\hskip-0.48em E}} X^T X / n$ is non-singular and write its inverse as $\Theta^0 := \Sigma_0^{-1}$. The parameter of interest is the first component $\beta_1^0$ of $\beta^0$. We show that the asymptotic variance of a de-biased Lasso estimator can be smaller than $\Theta_{1,1}^0$, under the conditions: $\beta^0$ is sparse in the sense that it has $s_0 = o(\sqrt n / \log p)$ non-zero entries and the first column $\Theta_1^0$ of $\Theta^0$ is not sparse. As by-product, we obtain some results for the Lasso estimator of $\beta^0$ for cases where $\beta^0$ is not sparse.