Coupled Markov chain Monte Carlo for high-dimensional regression with Half-t priors

Continuous shrinkage priors are commonly used in Bayesian analysis of high-dimensional data, due to both their computational advantages and favorable statistical properties. We develop coupled Markov chain Monte Carlo (MCMC) algorithms for Bayesian shrinkage regression in high dimensions. Following Glynn & Rhee (2014), these couplings can then be used in parallel computation strategies and practical diagnostics of convergence. Focusing on a class of shrinkage priors which include the Horseshoe, we demonstrate the scalability of the proposed couplings with high-dimensional simulations and data from a genome-wide association study with 2000 rows and 100,000 covariates. The results highlight the impact of the shrinkage prior on the computational efficiency of the coupling procedure, and motivates priors where the local precisions are Half-t distributions with degree of freedom larger than one, which are statistically justifiable in terms of posterior concentration, and lead to practical computational costs.