Positive factor networks: A graphical framework for modeling non-negative sequential data

We present a framework for modeling sequential data with hierarchical structure as a system of coupled non-negative vector factorization equations. We propose a graphical modeling representation and present algorithms for performing inference and learning that leverage existing non-negative matrix factorization (NMF) algorithms. The model variables satisfy the additive linearity property, so that an additive combination of representable configurations of the variables is also representable by the model. This property appears to make our approach well suited to modeling the magnitude spectra of audio signals and could potentially lead to useful applications in the area of computational auditory scene analysis. The proposed inference and learning algorithms are straightforward to implement and we present empirical results to illustrate their robustness. We present results on synthetic data sets for a regular expression example and a multiple target tracking example. We present an example of sparse hierarchical sequence decomposition on a real-world data set consisting of magnitude spectrogram data. Such a decomposition may be useful for audio compression and transcription applications. We also propose a network for language modeling.