Momentum-Based Variance Reduction in Non-Convex SGD

24 May 2019

Papers citing "Momentum-Based Variance Reduction in Non-Convex SGD"

27 / 77 papers shown

Title
SADAM: Stochastic Adam, A Stochastic Operator for First-Order Gradient-based Optimizer Wei Zhang Yun-Jian Bao ODL 25 2 0 20 May 2022
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction Junyi Li Feihu Huang Heng-Chiao Huang FedML 19 27 0 03 May 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 25 1 0 28 Apr 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 33 179 0 28 Mar 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 20 2 0 21 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 26 8 0 18 Feb 2022
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization Wei Jiang Bokun Wang Yibo Wang Lijun Zhang Tianbao Yang 76 17 0 15 Feb 2022
PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation Matilde Gargiani Andrea Zanelli Andrea Martinelli Tyler H. Summers John Lygeros 33 14 0 01 Feb 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin R. L. Jin Tianbao Yang 39 46 0 07 Dec 2021
Training Structured Neural Networks Through Manifold Identification and Variance Reduction Zih-Syuan Huang Ching-pei Lee AAML 46 9 0 05 Dec 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 23 7 0 26 Nov 2021
Linear Speedup in Personalized Collaborative Learning El Mahdi Chayti Sai Praneeth Karimireddy Sebastian U. Stich Nicolas Flammarion Martin Jaggi FedML 15 13 0 10 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 35 1 0 25 Oct 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang Kaipeng Zhang Tamer Bacsar 33 57 0 12 Oct 2021
Enhanced Bilevel Optimization via Bregman Distance Feihu Huang Junyi Li Shangqian Gao Heng-Chiao Huang 24 33 0 26 Jul 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 74 62 0 23 Jul 2021
A Survey on Fault-tolerance in Distributed Optimization and Machine Learning Shuo Liu AI4CE OOD 47 13 0 16 Jun 2021
Provably Faster Algorithms for Bilevel Optimization Junjie Yang Kaiyi Ji Yingbin Liang 46 132 0 08 Jun 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark W. Schmidt Simon Lacoste-Julien 18 17 0 18 Feb 2021
Learning from History for Byzantine Robust Optimization Sai Praneeth Karimireddy Lie He Martin Jaggi FedML AAML 24 173 0 18 Dec 2020
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 35 82 0 07 Dec 2020
Single-Timescale Stochastic Nonconvex-Concave Optimization for Smooth Nonlinear TD Learning Shuang Qiu Zhuoran Yang Xiaohan Wei Jieping Ye Zhaoran Wang 31 38 0 23 Aug 2020
Riemannian stochastic recursive momentum method for non-convex optimization Andi Han Junbin Gao ODL 20 16 0 11 Aug 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 82 53 0 24 Jun 2020
An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives Qi Qi Zhishuai Guo Yi Tian Xu R. L. Jin Tianbao Yang 31 44 0 17 Jun 2020
A Unified Convergence Analysis for Shuffling-Type Gradient Methods Lam M. Nguyen Quoc Tran-Dinh Dzung Phan Phuong Ha Nguyen Marten van Dijk 39 78 0 19 Feb 2020
Demon: Improved Neural Network Training with Momentum Decay John Chen Cameron R. Wolfe Zhaoqi Li Anastasios Kyrillidis ODL 24 15 0 11 Oct 2019