Bayesian Conditional Density Filtering for Big Data

We propose a Conditional Density Filtering (C-DF) algorithm for efficient online Bayesian inference. C-DF adapts Gibbs sampling to the online setting, sampling from approximations to conditional posterior distributions obtained by tracking of surrogate conditional sufficient statistics as new data arrive. This tracking eliminates the need to store or process the entire data set simultaneously. We show that C-DF samples converge to the exact posterior distribution asymptotically, as sampling proceeds and more data arrive over time. We provide several motivating examples, and consider an application to compressed factor regression for streaming data, illustrating competitive performance with batch algorithms that use all of the data.