Algebra of inference in graphical models revisited

Graphical Models use the intuitive and well-studied methods of graph theory to implicitly represent dependencies between variables in large systems and model the global behaviour of a complex system by specifying only local factors. The variational perspective poses inference as optimization and provides a rich framework to study inference when the object of interest is a (log) probability. However, graphical models can operate on a much wider set of algebraic structures. This paper builds on the work of Aji and McEliece (2000), to formally and broadly express what constitutes an inference problem in a graphical model. We then study the computational complexity of inference and show that inference in any commutative semiring is NP-hard under randomized reduction. By confining inference to four basic operations of min, max, sum and product, we introduce the inference hierarchy with an eye on computational complexity and establish the limits of message passing using distributive law in solving the problems in this hierarchy.