Automatic Relevance Determination in Nonnegative Matrix Factorization with the beta-Divergence

This paper addresses the problem of estimating the latent dimensionality in nonnegative matrix factorization (NMF). The estimation is done via automatic relevance determination (ARD). Uncovering the model order is important as it is necessary to strike the right balance between data fidelity and overfitting. We propose a Bayesian model for NMF and two families of algorithms known as l1-ARD and l2-ARD, each assuming different priors on the basis elements and the activation coefficients. The proposed algorithms leverage on the recent algorithmic advances in NMF with the beta-divergence using majorization-minimization (MM) methods. They are parametrized by the shape parameter beta which governs the assumption on the noise statistics. We show by deriving various auxiliary functions that the cost functions of the algorithms decrease monotonically to a local minimum. We also describe a heuristic to select the hyperparameters based on the method of moments. We demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data, the swimmer dataset, a music decomposition example and a stock price prediction task.