Towards Understanding Sharpness-Aware Minimization

13 June 2022

Papers citing "Towards Understanding Sharpness-Aware Minimization"

50 / 100 papers shown

Title
Sharpness-Aware Minimization and the Edge of Stability Philip M. Long Peter L. Bartlett AAML 25 9 0 21 Sep 2023
Adversarial Collaborative Filtering for Free Huiyuan Chen Xiaoting Li Vivian Lai Chin-Chia Michael Yeh Yujie Fan Yan Zheng Mahashweta Das Hao Yang AAML 15 6 0 20 Aug 2023
DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning Qinglun Li Li Shen Guang-Ming Li Quanjun Yin Dacheng Tao FedML 26 7 0 16 Aug 2023
Lookbehind-SAM: k steps back, 1 step forward Gonçalo Mordido Pranshu Malviya A. Baratin Sarath Chandar AAML 33 1 0 31 Jul 2023
Sharpness-Aware Graph Collaborative Filtering Huiyuan Chen Chin-Chia Michael Yeh Yujie Fan Yan Zheng Junpeng Wang Vivian Lai Mahashweta Das Hao Yang 18 5 0 18 Jul 2023
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin AAML 22 20 0 15 Jul 2023
FAM: Relative Flatness Aware Minimization Linara Adilova Amr Abourayya Jianning Li Amin Dada Henning Petzka Jan Egger Jens Kleesiek Michael Kamp ODL 8 1 0 05 Jul 2023
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization Khashayar Gatmiry Zhiyuan Li Ching-Yao Chuang Sashank J. Reddi Tengyu Ma Stefanie Jegelka ODL 11 11 0 22 Jun 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
Differentially Private Sharpness-Aware Training Jinseong Park Hoki Kim Yujin Choi Jaewook Lee 11 8 0 09 Jun 2023
Normalization Layers Are All That Sharpness-Aware Minimization Needs Maximilian Mueller Tiffany J. Vlaar David Rolnick Matthias Hein 8 18 0 07 Jun 2023
Optimal Transport Model Distributional Robustness Van-Anh Nguyen Trung Le Anh Tuan Bui Thanh-Toan Do Dinh Q. Phung OOD 17 3 0 07 Jun 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 34 15 0 05 Jun 2023
Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing Hye-jin Shim Jee-weon Jung Tomi Kinnunen 6 13 0 31 May 2023
Sharpness-Aware Minimization Leads to Low-Rank Features Maksym Andriushchenko Dara Bahri H. Mobahi Nicolas Flammarion AAML 25 25 0 25 May 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 21 17 0 24 May 2023
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training Yi Shi Yingqi Liu Yan Sun Zihao Lin Li Shen Xueqian Wang Dacheng Tao FedML 26 10 0 24 May 2023
Sharpness-Aware Data Poisoning Attack Pengfei He Han Xu J. Ren Yingqian Cui Hui Liu Charu C. Aggarwal Jiliang Tang AAML 32 7 0 24 May 2023
Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization Z. Fu Yixuan Su Zaiqiao Meng Nigel Collier MedIm 19 3 0 22 May 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 22 6 0 18 May 2023
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy Yi Shi Kang Wei Li Shen Yingqi Liu Xueqian Wang Bo Yuan Dacheng Tao FedML 26 2 0 01 May 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 79 31 0 28 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 22 39 0 07 Apr 2023
Per-Example Gradient Regularization Improves Learning Signals from Noisy Data Xuran Meng Yuan Cao Difan Zou 22 5 0 31 Mar 2023
Make Landscape Flatter in Differentially Private Federated Learning Yi Shi Yingqi Liu Kang Wei Li Shen Xueqian Wang Dacheng Tao FedML 17 52 0 20 Mar 2023
Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach Zhimeng Jiang Xiaotian Han Hongye Jin Guanchu Wang Rui Chen Na Zou Xia Hu 8 13 0 06 Mar 2023
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks Hao Sun Li Shen Qihuang Zhong Liang Ding Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao 41 31 0 01 Mar 2023
On Statistical Properties of Sharpness-Aware Minimization: Provable Guarantees Kayhan Behdin Rahul Mazumder 19 5 0 23 Feb 2023
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy Yan Sun Li Shen Tiansheng Huang Liang Ding Dacheng Tao FedML 29 50 0 21 Feb 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 13 7 0 19 Feb 2023
Approximately Bayes-Optimal Pseudo Label Selection Julian Rodemann Jann Goschenhofer Emilio Dorigatti T. Nagler Thomas Augustin 16 8 0 17 Feb 2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon Atish Agarwala Yann N. Dauphin 17 20 0 17 Feb 2023
A Modern Look at the Relationship between Sharpness and Generalization Maksym Andriushchenko Francesco Croce Maximilian Müller Matthias Hein Nicolas Flammarion 3DH 11 52 0 14 Feb 2023
Improving the Model Consistency of Decentralized Federated Learning Yi Shi Li Shen Kang Wei Yan Sun Bo Yuan Xueqian Wang Dacheng Tao FedML 15 51 0 08 Feb 2023
Exploring the Effect of Multi-step Ascent in Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Woojin Lee Jaewook Lee 11 9 0 27 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurélien Lucchi 14 13 0 19 Jan 2023
Stability Analysis of Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Jaewook Lee 17 12 0 16 Jan 2023
Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta D. Durfee Ayan Acharya S. Keerthi Rahul Mazumder AAML 15 5 0 07 Dec 2022
Efficient Generalization Improvement Guided by Random Weight Perturbation Tao Li Wei Yan Zehao Lei Yingwen Wu Kun Fang Ming Yang X. Huang AAML 27 6 0 21 Nov 2022
SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness Gonçalo Mordido Sébastien Henwood Sarath Chandar Franccois Leduc-Primeau AAML 17 0 0 18 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 21 47 0 10 Nov 2022
SADT: Combining Sharpness-Aware Minimization with Self-Distillation for Improved Model Generalization Masud An Nur Islam Fahim Jani Boutellier 17 0 0 01 Nov 2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias Ryo Karakida Tomoumi Takase Tomohiro Hayase Kazuki Osawa 11 14 0 06 Oct 2022
CrAM: A Compression-Aware Minimizer Alexandra Peste Adrian Vladu Eldar Kurtic Christoph H. Lampert Dan Alistarh 13 8 0 28 Jul 2022
On the Maximum Hessian Eigenvalue and Generalization Simran Kaur Jérémy E. Cohen Zachary Chase Lipton 13 41 0 21 Jun 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 6 58 0 01 Feb 2022
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 32 22 0 07 Oct 2021
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization Xubo Yue Maher Nouiehed Raed Al Kontar ODL 12 4 0 10 Nov 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 119 1,190 0 16 Aug 2016