Towards Quantifying the Preconditioning Effect of Adam

Towards Quantifying the Preconditioning Effect of Adam

11 February 2024

Sujay Sanghavi

Inderjit S. Dhillon

ArXiv (abs)PDF HTML

Papers citing "Towards Quantifying the Preconditioning Effect of Adam"

5 / 5 papers shown

Title
LightSAM: Parameter-Agnostic Sharpness-Aware Minimization Yifei Cheng Li Shen Hao Sun Nan Yin Xiaochun Cao Enhong Chen AAML 28 0 0 30 May 2025
Towards Quantifying the Hessian Structure of Neural Networks Zhaorui Dong Yushun Zhang Zhi-Quan Luo Jianfeng Yao Ruoyu Sun 69 1 0 05 May 2025
Gradient Methods with Online Scaling Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 78 4 0 04 Nov 2024
Adam-mini: Use Fewer Learning Rates To Gain More Yushun Zhang Congliang Chen Ziniu Li Tian Ding Chenwei Wu Yinyu Ye Zhi-Quan Luo Ruoyu Sun 124 58 0 24 Jun 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Boyao Wang Tong Zhang 57 6 0 21 Jun 2024