Non-negative Tensor Mixture Learning for Discrete Density Estimation

We present an expectation-maximization (EM) based unified framework for non-negative tensor decomposition that optimizes the Kullback-Leibler divergence. To avoid iterations in each M-step and learning rate tuning, we establish a general relationship between low-rank decompositions and many-body approximations. Using this connection, we exploit that the closed-form solution of the many-body approximation updates all parameters simultaneously in the M-step. Our framework offers not only a unified methodology for a variety of low-rank structures, including CP, Tucker, and Tensor Train decompositions, but also their mixtures. Notably, the weights of each low-rank tensor in the mixture can be learned from the data, which enables us to leverage the advantage of different low-rank structures without careful selection of the structure in advance. We empirically demonstrate that our framework overall provides superior generalization in terms of discrete density estimation and classification when compared to conventional tensor-based approaches.

@article{ghalamkari2025_2405.18220,
  title={ Non-negative Tensor Mixture Learning for Discrete Density Estimation },
  author={ Kazu Ghalamkari and Jesper Løve Hinrich and Morten Mørup },
  journal={arXiv preprint arXiv:2405.18220},
  year={ 2025 }
}