v1v2 (latest)

Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization

13 February 2020

Papers citing "Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization"

35 / 35 papers shown

Title
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 89 0 0 16 May 2025
Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum Yuan Zhou Xinli Shi Xuelong Li Jiachen Zhong G. Wen Jinde Cao FedML 103 0 0 17 Apr 2025
Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum Keisuke Kamo Hideaki Iiduka 129 0 0 15 Jan 2025
On the Performance Analysis of Momentum Method: A Frequency Domain Perspective Xianliang Li Jun Luo Zhiwei Zheng Hanxiao Wang Li Luo Lingkun Wen Linlong Wu Sheng Xu 183 0 0 29 Nov 2024
Empirical Tests of Optimization Assumptions in Deep Learning Hoang Tran Qinzi Zhang Ashok Cutkosky 68 2 0 01 Jul 2024
Towards Exact Gradient-based Training on Analog In-memory Computing Zhaoxian Wu Tayfun Gokmen Malte J. Rasch Tianyi Chen 71 2 0 18 Jun 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination Simon Weissmann Sara Klein Waïss Azizian Leif Döring 89 3 0 22 May 2024
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 67 4 0 16 May 2024
Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks Hristo Papazov Scott Pesme Nicolas Flammarion 79 7 0 08 Mar 2024
Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity Wenzhi Gao Qi Deng 61 1 0 25 Jan 2024
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise Boyao Wang Yuxing Liu Xiaoyu Wang Tong Zhang 37 5 0 22 Dec 2023
High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise Yusu Hong Junhong Lin 67 9 0 03 Nov 2023
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality Kejie Tang Weidong Liu Yichen Zhang Xi Chen 50 2 0 28 May 2023
A Unified Momentum-based Paradigm of Decentralized SGD for Non-Convex Models and Heterogeneous Data Haizhou Du Chengdong Ni 38 1 0 01 Mar 2023
Delayed Stochastic Algorithms for Distributed Weakly Convex Optimization W. Gao Qinhao Deng 70 0 0 30 Jan 2023
Policy Gradient in Robust MDPs with Global Convergence Guarantee Qiuhao Wang C. Ho Marek Petrik 107 29 0 20 Dec 2022
Towards understanding how momentum improves generalization in deep learning Samy Jelassi Yuanzhi Li ODL MLT AI4CE 90 38 0 13 Jul 2022
Convergence of First-Order Methods for Constrained Nonconvex Optimization with Dependent Data Ahmet Alacaoglu Hanbaek Lyu 59 4 0 29 Mar 2022
Personalized incentives as feedback design in generalized Nash equilibrium problems F. Fabiani Andrea Simonetto Paul Goulart 62 5 0 24 Mar 2022
On Almost Sure Convergence Rates of Stochastic Gradient Methods Jun Liu Ye Yuan 97 38 0 09 Feb 2022
On the Convergence of mSGD and AdaGrad for Stochastic Optimization Ruinan Jin Yu Xing Xingkang He 53 11 0 26 Jan 2022
Convergence of an Asynchronous Block-Coordinate Forward-Backward Algorithm for Convex Composite Optimization Cheik Traoré Saverio Salzo S. Villa 24 0 0 14 Jan 2022
Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis Jikai Jin Samir Bhatt Haiyang Wang Liwei Wang 82 51 0 24 Oct 2021
Distributed stochastic inertial-accelerated methods with delayed derivatives for nonconvex problems Yangyang Xu Yibo Xu Yonggui Yan Jiewei Chen 66 4 0 24 Jul 2021
Robust Regression via Model Based Methods Armin Moharrer Khashayar Kamran E. Yeh Stratis Ioannidis 27 0 0 20 Jun 2021
Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimization Qi Deng Wenzhi Gao 70 14 0 06 Jun 2021
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization Xiaoyu Wang M. Johansson 62 2 0 05 Jun 2021
OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression Nicola Bastianello Andrea Simonetto E. Dall’Anese 60 3 0 27 May 2021
Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-size Rahul Madhavan Hemant Makwana 48 0 0 15 Apr 2021
Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness Vien V. Mai M. Johansson 90 40 0 12 Feb 2021
Minibatch optimal transport distances; analysis and applications Kilian Fatras Younes Zine Szymon Majewski Rémi Flamary Rémi Gribonval Nicolas Courty OT 116 60 0 05 Jan 2021
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance Anas Barakat Pascal Bianchi W. Hachem S. Schechtman 91 13 0 07 Dec 2020
A Modular Analysis of Provable Acceleration via Polyak's Momentum: Training a Wide ReLU Network and a Deep Linear Network Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy 76 24 0 04 Oct 2020
Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization Aaron Defazio 86 23 0 01 Oct 2020
Convergence of adaptive algorithms for weakly convex constrained optimization Ahmet Alacaoglu Yura Malitsky Volkan Cevher 61 10 0 11 Jun 2020