Stochastic Structured Mean-Field Variational Inference

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly. The algorithm relies heavily on the use of fully factorized variational distributions. However, this "mean-field" independence approximation introduces bias. We show how to relax the mean-field approximation to allow arbitrary dependences between global parameters and local hidden variables, reducing both bias and sensitivity to local optima.