An Inexact Proximal Alternating Direction Method for Non-convex and Non-smooth Matrix Factorization and Beyond

Since Non-convex and Non-smooth Matrix Factorization (NNMF) problems have great realistic significance in applications, they attract extensive attention in the fields of image processing and machine learning. We in this paper propose an inexact proximal alternating direction (IPAD) method for solving various complex NNMF problems. Our IPAD method is not a single algorithm, but a general and flexible framework which can fuse various numerical methods into. With a special designed error condition, the convergence properties of IPAD are analyzed for a general formulation, and can be extended to a wider range of problems. Moreover, an implementation method for checking the inexactness criterion is theoretically analyzed, which is more valid than the previously naive criteria in practice. Our IPAD algorithm is applied to a widely-concerned sparse dictionary learning problem on both synthetic and real-world data. The experimental results with detailed analyses and discussions are given to verify the efficiency of IPAD method.