Why Does Sharpness-Aware Minimization Generalize Better Than SGD?

Why Does Sharpness-Aware Minimization Generalize Better Than SGD?

11 October 2023

Quanquan Gu

Papers citing "Why Does Sharpness-Aware Minimization Generalize Better Than SGD?"

12 / 12 papers shown

Title
Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization Samuel Schapiro Han Zhao 71 0 0 06 Dec 2024
Reweighting Local Mimina with Tilted SAM Tian Li Tianyi Zhou J. Bilmes 28 0 0 30 Oct 2024
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems Bingcong Li Liang Zhang Niao He 36 3 0 18 Oct 2024
Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate Hinata Harada Hideaki Iiduka 28 1 0 16 Sep 2024
Forget Sharpness: Perturbed Forgetting of Model Biases Within SAM Dynamics Ankit Vani Frederick Tung Gabriel L. Oliveira Hossein Sharifi-Noghabi AAML 31 0 0 10 Jun 2024
Why is SAM Robust to Label Noise? Christina Baek Zico Kolter Aditi Raghunathan NoLa AAML 33 9 0 06 May 2024
Friendly Sharpness-Aware Minimization Tao Li Pan Zhou Zhengbao He Xinwen Cheng Xiaolin Huang AAML 41 15 0 19 Mar 2024
Benign overfitting in leaky ReLU networks with moderate input dimension Kedar Karhadkar Erin E. George Michael Murray Guido Montúfar Deanna Needell MLT 28 2 0 11 Mar 2024
SURE: SUrvey REcipes for building reliable and robust deep networks Yuting Li Yingyi Chen Xuanlong Yu Dexiong Chen Xi Shen UQCV OOD 31 4 0 01 Mar 2024
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 63 34 0 04 Oct 2022
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 119 97 0 16 Oct 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 83 98 0 13 Oct 2021