Posterior Regularization on Bayesian Hierarchical Mixture Clustering

Bayesian hierarchical mixture clustering (BHMC) is an interesting model that improves on the traditional Bayesian hierarchical clustering approaches. Regarding the parent-to-node diffusion in the generative process, BHMC replaces the conventional Gaussian-to-Gaussian (G2G) kernels with a Hierarchical Dirichlet Process Mixture Model (HDPMM). However, the drawback of the BHMC lies in that it might obtain comparatively high nodal variance in the higher levels (i.e., those closer to the root node). This can be interpreted as that the separation between the nodes, in particular those in the higher levels, might be weak. Attempting to overcome this drawback, we consider a recent inferential framework named posterior regularization, which facilitates a simple manner to impose extra constraints on a Bayesian model to address some weakness of the original model. Hence, to enhance the separation of clusters, we apply posterior regularization to impose max-margin constraints on the nodes at every level of the hierarchy. In this paper, we illustrate how the framework integrates with the BHMC and achieves the desired improvements over the original model.