Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection

Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance matrix that can be decomposed into a sparse matrix minus a low-rank matrix from sample data. Solving this convex optimization problem is very challenging, especially for large problems. In this paper, we propose a novel alternating direction method of multipliers (ADMM) for solving this problem. The classical ADMM does not apply to this problem because there are three blocks in the problem and there is currently no convergence guarantee. Our method is a variant of the classical ADMM but only consists of two blocks and one of the subproblems is solved inexactly. Our method exploits and takes advantage of the special structure of the problem and thus can solve large problems very efficiently. Global convergence result is established for our proposed method. Numerical results on both synthetic data and gene expression data show that our method usually solve problems with one million variables in one to two minutes, and are usually five to thirty five times faster than a state-of-the-art Newton-CG proximal point algorithm.