Sharpness-Aware Minimization for Efficiently Improving Generalization

3 October 2020

Papers citing "Sharpness-Aware Minimization for Efficiently Improving Generalization"

50 / 867 papers shown

Title
SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes Nicolas Larue Ngoc-Son Vu Vitomir Štruc Peter Peer V. Christophides AAML 35 27 0 21 Nov 2022
Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning Zenglin Shi Jing Jie Ying Sun J. Lim Mengmi Zhang CLL 36 1 0 21 Nov 2022
Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation Jiawei Du Yiding Jiang Vincent Y. F. Tan Qiufeng Wang Haizhou Li DD 35 109 0 20 Nov 2022
SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness Gonçalo Mordido Sébastien Henwood Sarath Chandar Franccois Leduc-Primeau AAML 25 0 0 18 Nov 2022
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization Hiroki Naganuma Kartik Ahuja S. Takagi Tetsuya Motokawa Rio Yokota Kohta Ishikawa I. Sato Ioannis Mitliagkas OOD 13 7 0 15 Nov 2022
Instance-aware Model Ensemble With Distillation For Unsupervised Domain Adaptation Weimin Wu Jiayuan Fan Tao Chen Hancheng Ye Bo-Wen Zhang Baopu Li 18 3 0 15 Nov 2022
Towards A Unified Conformer Structure: from ASR to ASV Task Dexin Liao Tao Jiang Feng Wang Lin Li Q. Hong 27 10 0 14 Nov 2022
Accounting for Temporal Variability in Functional Magnetic Resonance Imaging Improves Prediction of Intelligence Yongqian Li Xin Ma Rajshekhar Sunderraman Shihao Ji Suprateek Kundu 22 6 0 11 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 23 47 0 10 Nov 2022
When is Momentum Extragradient Optimal? A Polynomial-Based Analysis J. Kim Gauthier Gidel Anastasios Kyrillidis Fabian Pedregosa 27 1 0 09 Nov 2022
Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning Zafir Stojanovski Karsten Roth Zeynep Akata 18 16 0 06 Nov 2022
SADT: Combining Sharpness-Aware Minimization with Self-Distillation for Improved Model Generalization Masud An Nur Islam Fahim Jani Boutellier 34 0 0 01 Nov 2022
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization Xiang Li Junchi Yang Niao He 26 8 0 31 Oct 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo-Lu Zhao I. Ganev Robin G. Walters Rose Yu Nima Dehmamy 47 16 0 31 Oct 2022
Introducing topography in convolutional neural networks Maxime Poli Emmanuel Dupoux Rachid Riad 23 0 0 28 Oct 2022
Fully-attentive and interpretable: vision and video vision transformers for pain detection Giacomo Fiorentini Itir Onal Ertugrul A. A. Salah MedIm ViT 11 2 0 27 Oct 2022
Watermarking for Out-of-distribution Detection Qizhou Wang Feng Liu Yonggang Zhang Jing Zhang Chen Gong Tongliang Liu Bo Han OODD 22 31 0 27 Oct 2022
UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification Yanbo Xu Alind Khare Glenn Matlin Monish Ramadoss Rishikesan Kamaleswaran Chao Zhang Alexey Tumanov 25 3 0 26 Oct 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 40 49 0 25 Oct 2022
Sufficient Invariant Learning for Distribution Shift Taero Kim Sungjun Lim Kyungwoo Song OOD 31 2 0 24 Oct 2022
K-SAM: Sharpness-Aware Minimization at the Speed of SGD Renkun Ni Ping Yeh-Chiang Jonas Geiping Micah Goldblum A. Wilson Tom Goldstein 20 8 0 23 Oct 2022
Large-batch Optimization for Dense Visual Predictions Zeyue Xue Jianming Liang Guanglu Song Zhuofan Zong Liang Chen Yu Liu Ping Luo VLM 33 9 0 20 Oct 2022
Rethinking Sharpness-Aware Minimization as Variational Inference Szilvia Ujváry Zsigmond Telek A. Kerekes Anna Mészáros Ferenc Huszár 30 8 0 19 Oct 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models Nikolaos Dimitriadis P. Frossard Franccois Fleuret 26 25 0 18 Oct 2022
5th Place Solution to Kaggle Google Universal Image Embedding Competition Noriaki Ota Shingo Yokoi Shinsuke Yamaoka 118 2 0 18 Oct 2022
SQuAT: Sharpness- and Quantization-Aware Training for BERT Zheng Wang Juncheng Billy Li Shuhui Qu Florian Metze Emma Strubell MQ 21 7 0 13 Oct 2022
GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization Zhiyuan Zhang Ruixuan Luo Qi Su Xueting Sun 29 11 0 13 Oct 2022
Improving the Reliability for Confidence Estimation Haoxuan Qu Yanchao Li Lin Geng Foo Jason Kuen Jiuxiang Gu Jun Liu UQCV 21 9 0 13 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 31 47 0 13 Oct 2022
Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation Zeyu Qin Yanbo Fan Yi Liu Li Shen Yong Zhang Jue Wang Baoyuan Wu AAML SILM 26 79 0 12 Oct 2022
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models Qihuang Zhong Liang Ding Li Shen Peng Mi Juhua Liu Bo Du Dacheng Tao AAML 28 50 0 11 Oct 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 45 56 0 11 Oct 2022
Meta-Learning with Self-Improving Momentum Target Jihoon Tack Jongjin Park Hankook Lee Jaeho Lee Jinwoo Shin LRM 60 12 0 11 Oct 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 27 69 0 11 Oct 2022
Second-order regression models exhibit progressive sharpening to the edge of stability Atish Agarwala Fabian Pedregosa Jeffrey Pennington 25 26 0 10 Oct 2022
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul F. Chen Brett W. Larsen Jonathan Frankle Surya Ganguli Gintare Karolina Dziugaite UQCV 25 38 0 06 Oct 2022
SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data Ching-Yun Ko Pin-Yu Chen Jeet Mohapatra Payel Das Lucani E. Daniel 21 3 0 06 Oct 2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias Ryo Karakida Tomoumi Takase Tomohiro Hayase Kazuki Osawa 13 14 0 06 Oct 2022
Invariant Aggregator for Defending against Federated Backdoor Attacks Xiaoya Wang Dimitrios Dimitriadis Oluwasanmi Koyejo Shruti Tople FedML 34 1 0 04 Oct 2022
MEDFAIR: Benchmarking Fairness for Medical Imaging Yongshuo Zong Yongxin Yang Timothy M. Hospedales OOD 76 58 0 04 Oct 2022
SAM as an Optimal Relaxation of Bayes Thomas Möllenhoff Mohammad Emtiyaz Khan BDL 31 32 0 04 Oct 2022
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 73 34 0 04 Oct 2022
pMPL: A Robust Multi-Party Learning Framework with a Privileged Party Lushan Song Jiaxuan Wang Zhexuan Wang Xinyu Tu Guopeng Lin Wenqiang Ruan Haoqi Wu Wei Han 19 18 0 02 Oct 2022
Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability Alexandru Damian Eshaan Nichani Jason D. Lee 22 76 0 30 Sep 2022
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel Sungyub Kim Si-hun Park Kyungsu Kim Eunho Yang BDL 26 4 0 30 Sep 2022
Stop Wasting My Time! Saving Days of ImageNet and BERT Training with Latest Weight Averaging Jean Kaddour MoMe 3DH 24 39 0 29 Sep 2022
Learning Gradient-based Mixup towards Flatter Minima for Domain Generalization Danni Peng Sinno Jialin Pan 34 2 0 29 Sep 2022
Colonoscopy Landmark Detection using Vision Transformers Aniruddha Tamhane Tseéla Mida Erez Posner Moshe Bouhnik ViT MedIm 56 5 0 22 Sep 2022
Periodic Extrapolative Generalisation in Neural Networks Peter Belcak Roger Wattenhofer 6 3 0 21 Sep 2022
Deep Double Descent via Smooth Interpolation Matteo Gamba Erik Englesson Marten Bjorkman Hossein Azizpour 63 10 0 21 Sep 2022