Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings

We propose two new classes of prior densities for Bayesian model selection, and show that selection procedures based on priors from these classes are consistent for linear models even when the number of covariates $p$ increases sub-exponentially with the sample size $n$, provided that certain regularity constraints are satisfied. We also demonstrate that the resulting prior densities impose fully adaptive penalties on regression parameters, distinguishing our model selection procedure from existing $L_0$ penalized likelihood methods. To implement this framework, we propose a scalable algorithm called Simplified Shotgun Stochastic Search with Screening (S5) that efficiently explores high-dimensional model spaces. Compared to standard MCMC algorithms, S5 can dramatically speed the rate at which Bayesian model selection procedures identify high posterior probability models. Finally, we present a detailed comparison between the proposed procedure and the most commonly used alternative methods for variable selection using precision-recall curves and more standard simulation scenarios.