Logit stick-breaking priors for Bayesian density regression

There is an increasing focus in several fields on learning how the distribution of an outcome changes with a set of covariates. Bayesian nonparametric dependent mixture models provide a useful approach to flexibly address this goal, however many representations are characterized by difficult interpretation and intractable computational methods. Motivated by these issues, we describe a flexible formulation for Bayesian density regression, which is defined as a potentially infinite mixture model whose probability weights change with the covariates via a stick-breaking construction relying on a set of logistic regressions. We derive theoretical properties and show that our logit stick-breaking representation can be interpreted as a simple continuation-ratio logistic regression. This result facilitates derivation of three computational methods of routine use in Bayesian inference, covering Markov Chain Monte Carlo via Gibbs sampling, the Expectation Conditional Maximization algorithm for the estimation of posterior modes, and a Variational Bayes approach for scalable inference. The algorithms associated with these methods are analytically derived and made available online at https://github.com/tommasorigon/DLSBP. We additionally compare the three computational strategies in an application to the Old Faithful Geyser dataset.