Breathing $k$-Means

We propose a new algorithm for the $k$-means problem which repeatedly increases and decreases the number of centroids by $m$ in order to find an approximate solution. New centroids are inserted in areas where they will likely reduce the error. The subsequent removal of centroids is done such that the resulting raise in error is small. After each increase or decrease step standard $k$-means is performed. Termination is guaranteed by decrementing $m$ after each increase/decrease cycle unless the overall error was lowered. In experiments with Gaussian mixture distributions the new algorithm produced on average solutions several percent better than $k$-means++.