12
11

Scalable Spike-and-Slab

Abstract

Spike-and-slab priors are commonly used for Bayesian variable selection, due to their interpretability and favorable statistical properties. However, existing samplers for spike-and-slab posteriors incur prohibitive computational costs when the number of variables is large. In this article, we propose Scalable Spike-and-Slab (S3S^3), a scalable Gibbs sampling implementation for high-dimensional Bayesian regression with the continuous spike-and-slab prior of George and McCulloch (1993). For a dataset with nn observations and pp covariates, S3S^3 has order max{n2pt,np}\max\{ n^2 p_t, np \} computational cost at iteration tt where ptp_t never exceeds the number of covariates switching spike-and-slab states between iterations tt and t1t-1 of the Markov chain. This improves upon the order n2pn^2 p per-iteration cost of state-of-the-art implementations as, typically, ptp_t is substantially smaller than pp. We apply S3S^3 on synthetic and real-world datasets, demonstrating orders of magnitude speed-ups over existing exact samplers and significant gains in inferential quality over approximate samplers with comparable cost.

View on arXiv
Comments on this paper