Faster Randomized Primal-Dual Algorithms For Nonsmooth Composite Convex Minimization

We develop two novel randomized primal-dual algorithms to solve nonsmooth composite convex optimization problems. The first algorithm is fully randomized, i.e., it has parallel randomized updates on both primal and dual variables, while the second one is a semi-randomized scheme, which only has one randomized update on the primal (or dual) variable while using the full update for the other. Both algorithms achieve the best-known $\mathcal{O}(1/k)$ or $\mathcal{O}(1/k^2)$ convergence rates in expectation under either only convexity or strong convexity, respectively, where $k$ is the iteration counter. These rates can be obtained for both the primal and dual problems. With new parameter update rules, our algorithms can be boosted up to $\underline{o}(1/(k\sqrt{\log{k}}))$ or $\underline{o}(1/(k^2\sqrt{\log{k}}))$-rates in expectation, respectively (see definitions below). To the best of our knowledge, this is the first time such faster convergence rates are shown for randomized primal-dual methods. Finally, we verify our theoretical results via two numerical examples and compare them with state-of-the-arts.