SAM operates far from home: eigenvalue regularization as a dynamical phenomenon

17 February 2023

Papers citing "SAM operates far from home: eigenvalue regularization as a dynamical phenomenon"

20 / 20 papers shown

Title
Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning Yeoreum Lee Jinwook Jung Sungyong Baik MoMe 40 0 0 20 Apr 2025
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation Dahun Shin Dongyeop Lee Jinseok Chung Namhoon Lee ODL AAML 90 0 0 25 Feb 2025
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems Bingcong Li Liang Zhang Niao He 36 3 0 18 Oct 2024
Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training Zhanpeng Zhou Mingze Wang Yuchen Mao Bingrui Li Junchi Yan AAML 55 0 0 14 Oct 2024
A Universal Class of Sharpness-Aware Minimization Algorithms B. Tahmasebi Ashkan Soleymani Dara Bahri Stefanie Jegelka P. Jaillet AAML 39 2 0 06 Jun 2024
Does SGD really happen in tiny subspaces? Minhak Song Kwangjun Ahn Chulhee Yun 47 4 1 25 May 2024
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 30 3 0 30 Apr 2024
A Precise Characterization of SGD Stability Using Loss Surface Geometry Gregory Dexter Borja Ocejo S. Keerthi Aman Gupta Ayan Acharya Rajiv Khanna MLT 15 0 0 22 Jan 2024
Neglected Hessian component explains mysteries in Sharpness regularization Yann N. Dauphin Atish Agarwala Hossein Mobahi FAtt 21 3 0 19 Jan 2024
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression Xuxing Chen Krishnakumar Balasubramanian Promit Ghosal Bhavya Agrawalla 20 7 0 02 Oct 2023
Sharpness-Aware Minimization and the Edge of Stability Philip M. Long Peter L. Bartlett AAML 22 9 0 21 Sep 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 26 15 0 16 Jun 2023
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning Libin Zhu Chaoyue Liu Adityanarayanan Radhakrishnan M. Belkin 17 13 0 07 Jun 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 18 17 0 24 May 2023
On Statistical Properties of Sharpness-Aware Minimization: Provable Guarantees Kayhan Behdin Rahul Mazumder 14 5 0 23 Feb 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 13 7 0 19 Feb 2023
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 60 34 0 04 Oct 2022
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 119 82 0 16 Oct 2021
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 150 198 0 04 Mar 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,696 0 15 Sep 2016