Sharpness-Aware Minimization for Efficiently Improving Generalization

3 October 2020

Papers citing "Sharpness-Aware Minimization for Efficiently Improving Generalization"

50 / 867 papers shown

Title
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 32 11 0 14 Feb 2023
A Modern Look at the Relationship between Sharpness and Generalization Maksym Andriushchenko Francesco Croce Maximilian Müller Matthias Hein Nicolas Flammarion 3DH 11 54 0 14 Feb 2023
Revisiting Weighted Aggregation in Federated Learning with Neural Networks Zexi Li Tao R. Lin Xinyi Shang Chao-Xiang Wu FedML 42 59 0 14 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 64 352 0 13 Feb 2023
Improving Differentiable Architecture Search via Self-Distillation Xunyu Zhu Jian Li Yong Liu Weiping Wang 21 7 0 11 Feb 2023
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion Haotian Ju Dongyue Li Aneesh Sharma Hongyang R. Zhang 31 40 0 09 Feb 2023
Improving the Model Consistency of Decentralized Federated Learning Yi Shi Li Shen Kang Wei Yan Sun Bo Yuan Xueqian Wang Dacheng Tao FedML 33 51 0 08 Feb 2023
Geometric Perception based Efficient Text Recognition P.N.Deelaka D.R.Jayakodi D.Y.Silva 21 3 0 08 Feb 2023
Flat Seeking Bayesian Neural Networks Van-Anh Nguyen L. Vuong Hoang Phan Thanh-Toan Do Dinh Q. Phung Trung Le BDL 22 8 0 06 Feb 2023
On a continuous time model of gradient descent dynamics and instability in deep learning Mihaela Rosca Yan Wu Chongli Qin Benoit Dherin 16 6 0 03 Feb 2023
AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge Coen de Vente Koen A. Vermeer Nicolas Jaccard He Wang Hongyi Sun ... Abdul Qayyum Imran Razzak Bram van Ginneken H. Lemij Clara I. Sánchez 21 55 0 03 Feb 2023
Simple, Effective and General: A New Backbone for Cross-view Image Geo-localization Yingying Zhu Hongji Yang Yuxin Lu Qiang Huang 19 31 0 03 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent Avrajit Ghosh He Lyu Xitong Zhang Rongrong Wang 50 20 0 02 Feb 2023
A Survey of Deep Learning: From Activations to Transformers Johannes Schneider Michalis Vlachos ViT MedIm AI4TS AI4CE 50 9 0 01 Feb 2023
A Comprehensive Survey of Continual Learning: Theory, Method and Application Liyuan Wang Xingxing Zhang Hang Su Jun Zhu KELM CLL 38 601 0 31 Jan 2023
Misspecification-robust Sequential Neural Likelihood for Simulation-based Inference Ryan P. Kelly David J. Nott David T. Frazier D. Warne Christopher C. Drovandi 25 10 0 31 Jan 2023
FedFA: Federated Feature Augmentation Tianfei Zhou E. Konukoglu OOD FedML 25 28 0 30 Jan 2023
Exploring the Effect of Multi-step Ascent in Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Woojin Lee Jaewook Lee 20 9 0 27 Jan 2023
Projected Subnetworks Scale Adaptation Siddhartha Datta N. Shadbolt VLM CLL 28 0 0 27 Jan 2023
Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers A. Vats Aman Chadha ViT 27 2 0 26 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 23 13 0 19 Jan 2023
$β$ -DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search Peng Ye Tong He Baopu Li Tao Chen Lei Bai Wanli Ouyang OOD 40 7 0 16 Jan 2023
Stability Analysis of Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Jaewook Lee 33 12 0 16 Jan 2023
WLD-Reg: A Data-dependent Within-layer Diversity Regularizer Firas Laakom Jenni Raitoharju Alexandros Iosifidis Moncef Gabbouj AI4CE 26 7 0 03 Jan 2023
GoogLe2Net: Going Transverse with Convolutions Yuanpeng He 16 2 0 01 Jan 2023
FlatENN: Train Flat for Enhanced Fault Tolerance of Quantized Deep Neural Networks Akul Malhotra S. Gupta 11 0 0 29 Dec 2022
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data Harsh Rangwani Sumukh K Aithal Mayank Mishra R. Venkatesh Babu 31 28 0 28 Dec 2022
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 31 17 0 27 Dec 2022
DSI++: Updating Transformer Memory with New Documents Sanket Vaibhav Mehta Jai Gupta Yi Tay Mostafa Dehghani Vinh Q. Tran J. Rao Marc Najork Emma Strubell Donald Metzler CLL 32 39 0 19 Dec 2022
Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint Borui Zhang Wenzhao Zheng Jie Zhou Jiwen Lu AAML 25 7 0 18 Dec 2022
Learning threshold neurons via the "edge of stability" Kwangjun Ahn Sébastien Bubeck Sinho Chewi Y. Lee Felipe Suarez Yi Zhang MLT 36 36 0 14 Dec 2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging Peng Lu I. Kobyzev Mehdi Rezagholizadeh Ahmad Rashid A. Ghodsi Philippe Langlais MoMe 33 11 0 12 Dec 2022
ezDPS: An Efficient and Zero-Knowledge Machine Learning Inference Pipeline Haodi Wang Thang Hoang 16 11 0 11 Dec 2022
Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks Yihan Wu Aleksandar Bojchevski Heng Huang AAML 34 30 0 09 Dec 2022
MixBoost: Improving the Robustness of Deep Neural Networks by Boosting Data Augmentation Zhendong Liu Wenyu Jiang Min Guo Chongjun Wang AAML 21 1 0 08 Dec 2022
Mitigating Memorization of Noisy Labels by Clipping the Model Prediction Hongxin Wei Huiping Zhuang Renchunzi Xie Lei Feng Gang Niu Bo An Yixuan Li VLM NoLa 24 29 0 08 Dec 2022
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning Kyuyong Shin Hanock Kwak Wonjae Kim Jisu Jeong Seungjae Jung KyungHyun Kim Jung-Woo Ha Sang-Woo Lee 24 4 0 07 Dec 2022
Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta D. Durfee Ayan Acharya S. Keerthi Rahul Mazumder AAML 28 5 0 07 Dec 2022
Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks Michal Jamro.z Marcin Kurdziel 14 0 0 01 Dec 2022
Context-Aware Robust Fine-Tuning Xiaofeng Mao YueFeng Chen Xiaojun Jia Rong Zhang Hui Xue Zhao Li VLM CLIP 35 24 0 29 Nov 2022
A Theoretical Study of Inductive Biases in Contrastive Learning Jeff Z. HaoChen Tengyu Ma UQCV SSL 33 31 0 27 Nov 2022
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices Kazuki Osawa Shigang Li Torsten Hoefler AI4CE 35 24 0 25 Nov 2022
Cross-Domain Ensemble Distillation for Domain Generalization Kyung-Jin Lee Sungyeon Kim Suha Kwak FedML OOD 26 38 0 25 Nov 2022
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 25 11 0 24 Nov 2022
Differentially Private Image Classification from Features Harsh Mehta Walid Krichene Abhradeep Thakurta Alexey Kurakin Ashok Cutkosky 52 7 0 24 Nov 2022
A Dual-scale Lead-seperated Transformer With Lead-orthogonal Attention And Meta-information For Ecg Classification Heng Chang Guijin Wang Zhourui Xia Wenming Yang Li Sun MedIm 29 1 0 23 Nov 2022
Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization Zifa Wang Nan Ding Tomer Levinboim Xi Chen Radu Soricut AAML 35 5 0 22 Nov 2022
Efficient Generalization Improvement Guided by Random Weight Perturbation Tao Li Wei Yan Zehao Lei Yingwen Wu Kun Fang Ming Yang X. Huang AAML 35 6 0 21 Nov 2022