The Role of Momentum Parameters in the Optimal Convergence of Adaptive
Polyak's Heavy-ball Methods

The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods

International Conference on Learning Representations (ICLR), 2021

15 February 2021

ArXiv (abs)PDF HTML

Papers citing "The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods"

5 / 5 papers shown

Title
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments M. S. Shamaee S. F. Hafshejani 108 1 0 03 Sep 2023
A Unified Momentum-based Paradigm of Decentralized SGD for Non-Convex Models and Heterogeneous Data Haizhou Du Chengdong Ni 88 3 0 01 Mar 2023
Adapting Step-size: A Unified Perspective to Analyze and Improve Gradient-based Methods for Adversarial Attacks Wei Tao Lei Bao Long Sheng Gao-wei Wu Qing Tao AAML 159 2 0 27 Jan 2023
Does Momentum Change the Implicit Regularization on Separable Data?Neural Information Processing Systems (NeurIPS), 2021 Bohan Wang Qi Meng Huishuai Zhang Tian Ding Wei Chen Zhirui Ma Tie-Yan Liu 194 23 0 08 Oct 2021
On the Last Iterate Convergence of Momentum MethodsInternational Conference on Algorithmic Learning Theory (ALT), 2021 Xiaoyun Li Mingrui Liu Francesco Orabona 258 12 0 13 Feb 2021