SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients

v1v2v3v4v5v6v7v8v9v10 (latest)

SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients

15 June 2021

Heng-Chiao Huang

ArXiv (abs)PDF HTML Github (17★)

Papers citing "SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients"

14 / 14 papers shown

Title
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 89 0 0 16 May 2025
Online Nonconvex Bilevel Optimization with Bregman Divergences Jason Bohne David Rosenberg Gary Kazantsev Pawel Polak 80 0 0 16 Sep 2024
Gradient-Free Method for Heavily Constrained Nonconvex Optimization Wanli Shi Hongchang Gao Bin Gu 88 5 0 31 Aug 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 120 13 0 06 Feb 2024
Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case Meixuan He Yuqing Liang Jinlan Liu Dongpo Xu 82 9 0 20 Jul 2023
Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization Feihu Huang Chunyu Xuan Xinrui Wang Siqi Zhang Songcan Chen 118 7 0 07 Mar 2023
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks Hao Sun Li Shen Qihuang Zhong Liang Ding Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao 98 34 0 01 Mar 2023
Faster Adaptive Momentum-Based Federated Methods for Distributed Composition Optimization Feihu Huang FedML 90 1 0 03 Nov 2022
Fast Adaptive Federated Bilevel Optimization Feihu Huang FedML 108 7 0 02 Nov 2022
META-STORM: Generalized Fully-Adaptive Variance Reduced SGD for Unbounded Functions Zijian Liu Ta Duy Nguyen Thien Hai Nguyen Alina Ene Huy Le Nguyen 88 6 0 29 Sep 2022
Provable Adaptivity of Adam under Non-uniform Smoothness Bohan Wang Yushun Zhang Huishuai Zhang Qi Meng Ruoyu Sun Zhirui Ma Tie-Yan Liu Zhimin Luo Wei Chen 77 26 0 21 Aug 2022
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 110 70 0 20 Aug 2022
A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse Junyi Li Bin Gu Heng-Chiao Huang 100 74 0 09 Dec 2021
Enhanced Bilevel Optimization via Bregman Distance Feihu Huang Junyi Li Shangqian Gao Heng-Chiao Huang 80 33 0 26 Jul 2021