Adam$^+$: A Stochastic Method with Adaptive Variance Reduction

Adam $^+$ : A Stochastic Method with Adaptive Variance Reduction

24 November 2020

Wei Zhang

Francesco Orabona

ArXiv (abs)PDF HTML

Papers citing "Adam$^+$: A Stochastic Method with Adaptive Variance Reduction"

16 / 16 papers shown

Title
An Adaptive Method Stabilizing Activations for Enhanced Generalization Hyunseok Seung Jaewoo Lee Hyunsuk Ko ODL 21 0 0 10 Jun 2025
Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism Sameera Ramasinghe Thalaiyasingam Ajanthan Gil Avraham Yan Zuo Alexander Long GNN 79 0 0 02 Jun 2025
Temporal Context Consistency Above All: Enhancing Long-Term Anticipation by Learning and Enforcing Temporal Constraints Alberto Maté Mariella Dimiccoli AI4TS 87 1 0 27 Dec 2024
Fast and Accurate Neural Rendering Using Semi-Gradients In-Young Cho Jaewoong Cho 52 0 0 14 Oct 2024
Dynamic Estimation of Learning Rates Using a Non-Linear Autoregressive Model Ramin Okhrati 27 0 0 13 Oct 2024
Gradient-Free Method for Heavily Constrained Nonconvex Optimization Wanli Shi Hongchang Gao Bin Gu 88 5 0 31 Aug 2024
The Implicit Bias of Adam on Separable Data Chenyang Zhang Difan Zou Yuan Cao AI4CE 94 9 0 15 Jun 2024
PanBench: Towards High-Resolution and High-Performance Pansharpening Shiying Wang Xuechao Zou Kai Li Junliang Xing Pin Tao 88 1 0 20 Nov 2023
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward Yunlong Tang Siting Xu Teng Wang Qin Lin Qinglin Lu Feng Zheng VOS 100 11 0 25 Sep 2022
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models Xingyu Xie Pan Zhou Huan Li Zhouchen Lin Shuicheng Yan ODL 94 169 0 13 Aug 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin Rong Jin Tianbao Yang 88 49 0 07 Dec 2021
SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients Feihu Huang Junyi Li Heng-Chiao Huang ODL 84 42 0 15 Jun 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark Schmidt Simon Lacoste-Julien 61 18 0 18 Feb 2021
Variance Reduction on General Adaptive Stochastic Mirror Descent Wenjie Li Zhanyu Wang Yichen Zhang Guang Cheng 73 4 0 26 Dec 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 217 168 0 03 Jul 2020
Beyond Worst-Case Analysis in Stochastic Approximation: Moment Estimation Improves Instance Complexity J.N. Zhang Hongzhou Lin Subhro Das S. Sra Ali Jadbabaie 32 1 0 08 Jun 2020