Negative Binomial Process Count and Mixture Modeling

The seemingly disjoint problems of count and mixture modeling are united under the negative binomial (NB) process. We reveal relationships between the Poisson, multinomial, gamma and Dirichlet distributions, and construct a Poisson-logarithmic bivariate count distribution that connects the NB and Chinese restaurant table distributions. Fundamental properties of the models are developed, and we derive efficient Bayesian inference. It is shown that with augmentation and normalization, the NB process and gamma-NB process can be reduced to the Dirichlet process and hierarchical Dirichlet process, respectively. These relationships highlight theoretical, structural and computational advantages of the NB process. A variety of NB processes including the beta-geometric, beta-NB, marked-beta-NB, marked-gamma-NB and zero-inflated-NB processes, with distinct sharing mechanisms, are also constructed. These models are applied to topic modeling, with connections made to existing algorithms under the Poisson factor analysis framework. Example results show the importance of inferring both the NB dispersion and probability parameters, which respectively govern the overdispersion level and variance-to-mean ratio for count modeling.