Clustering with Beta Divergences

Clustering algorithms start with a fixed divergence metric, which captures the possibly asymmetric distance between two samples. In a mixture model, the sample distribution plays the role of a divergence metric. It is often the case that the distributional assumption is not validated, which calls for an adaptive approach. We consider a richer model where the underlying distribution belongs to a parametrized exponential family, called Tweedie Models. We first show the connection between the Tweedie models and beta divergences, and derive the corresponding hard-assignment clustering algorithm. We exploit this connection to identify moment conditions and use Generalized Method of Moments(GMoM) to learn the data distribution. Based on this adaptive approach, we propose four new hard clustering algorithms and compare them to the classical k-means and DP-means on synthetic data as well as seven UCI datasets and one large gene expression dataset. We further compare the GMoM routine to an approximate maximum likelihood routine and validate the computational benefits of the GMoM approach.