Iterative Reweighted Singular Value Minimization Methods for $l_p$ Regularized Unconstrained Matrix Minimization

In this paper we study general $l_p$ regularized unconstrained matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them. And we show that the first-order stationary points introduced in related work for an $l_p$ regularized $vector$ minimization problem are equivalent to those of an $l_p$ regularized $matrix$ minimization reformulation. We also establish that any local minimizer of the $l_p$ regularized matrix minimization problems must be a first-order stationary point. Moreover, we derive lower bounds for nonzero singular values of the first-order stationary points and hence also of the local minimizers for the $l_p$ matrix minimization problems. The iterative reweighted singular value minimization (IRSVM) approaches are then proposed to solve these problems in which each subproblem has a closed-form solution. We show that any accumulation point of the sequence generated by these methods is a first-order stationary point of the problems. In addition, we study a nonmontone proximal gradient (NPG) method for solving the $l_p$ matrix minimization problems and establish its global convergence. Our computational results demonstrate that the IRSVM and NPG methods generally outperform some existing state-of-the-art methods in terms of solution quality and/or speed. Moreover, the IRSVM methods are slightly faster than the NPG method.