Ising Model Selection Using $\ell_{1}$-Regularized Linear Regression: A Statistical Mechanics Analysis

We theoretically investigate the typical learning performance of $\ell_{1}$-regularized linear regression ($\ell_1$-LinR) for Ising model selection using the replica method from statistical mechanics. For typical random regular (RR) graphs in the paramagnetic phase, an accurate estimate of the typical sample complexity of $\ell_1$-LinR is obtained, demonstrating that, for an Ising model with $N$ variables, $\ell_1$-LinR is model selection consistent with $M=\mathcal{O}\left(\log N\right)$ samples. Moreover, we provide a computationally efficient method to accurately predict the non-asymptotic behavior of $\ell_1$-LinR for moderate $M$ and $N$, such as the precision and recall rates. Simulations show a fairly good agreement between the theoretical predictions and experimental results, even for graphs with many loops, which supports our findings. Although this paper focuses on $\ell_1$-LinR, our method is readily applicable for precisely investigating the typical learning performances of a wide class of $\ell_{1}$-regularized M-estimators including $\ell_{1}$-regularized logistic regression and interaction screening.