Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium

Regularized inversion methods for image reconstruction are used widely due to their tractability and their ability to combine physical sensor models with useful regularity criteria. Such methods were used in the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoisers as regularizers in inversion. However, the need to formulate regularized inversion as the solution to an optimization problem limits the expressiveness of possible regularity conditions and the variety of provably convergent Plug-and-Play denoising operators. In this paper, we introduce the idea of Consensus Equilibrium (CE), which generalizes regularized inversion to include a wide variety of regularity operators without the need for an optimization formulation. CE is based on the solution of a set of equilibrium equations that balance data fit and regularity. In CE, the problem of MAP estimation in regularized inversion is replaced by the problem of solving these equations, which can be approached in multiple ways. The key contribution of CE is to provide a novel framework for fusing a wide variety of regularizing operators with physical sensor models, even for models and operators that are not expressible via optimization. We describe the derivation of the CE equations and prove that the solution of the CE equations generalizes the standard MAP estimate. We also discuss algorithms for solving the CE equations, including a version of the Douglas-Rachford (DR)/ADMM algorithm with a novel form of preconditioning and Newton's method, both standard form and a Jacobian-free form. We illustrate the idea of consensus equilibrium on several examples, one using an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually.