Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach

11 October 2022

Li Shen

Papers citing "Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach"

50 / 60 papers shown

Title
Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning Yeoreum Lee Jinwook Jung Sungyong Baik MoMe 40 0 0 20 Apr 2025
Layer-wise Adaptive Gradient Norm Penalizing Method for Efficient and Accurate Deep Learning Sunwoo Lee 98 0 0 18 Mar 2025
Sharpness-Aware Minimization: General Analysis and Improved Rates Dimitris Oikonomou Nicolas Loizou 65 0 0 04 Mar 2025
The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training Jinbo Wang Mingze Wang Zhanpeng Zhou Junchi Yan Weinan E Lei Wu 75 1 0 26 Feb 2025
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation Dahun Shin Dongyeop Lee Jinseok Chung Namhoon Lee ODL AAML 129 0 0 25 Feb 2025
Preconditioned Sharpness-Aware Minimization: Unifying Analysis and a Novel Learning Algorithm Yilang Zhang Bingcong Li G. Giannakis AAML 39 0 0 11 Jan 2025
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks Jinping Zou Xiaoge Deng Tao Sun 66 0 0 22 Dec 2024
SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM Xingyu Lyu Qianqian Xu Zhiyong Yang Shaojie Lyu Qingming Huang 64 0 0 18 Dec 2024
Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Aodi Li Liansheng Zhuang Xiao Long Minghong Yao Shafei Wang 130 0 0 18 Dec 2024
Enhancing generalization in high energy physics using white-box adversarial attacks Franck Rothen Samuel Klein Matthew Leigh T. Golling AAML 31 1 0 14 Nov 2024
Reweighting Local Mimina with Tilted SAM Tian Li Tianyi Zhou J. Bilmes 28 0 0 30 Oct 2024
Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognition Mengke Li Y. Liu Yang Lu Yiqun Zhang Yiu-ming Cheung Hui Huang VLM 31 2 0 28 Oct 2024
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems Bingcong Li Liang Zhang Niao He 36 3 0 18 Oct 2024
Combinatorial Multi-armed Bandits: Arm Selection via Group Testing Arpan Mukherjee Shashanka Ubaru K. Murugesan Karthikeyan Shanmugam A. Tajer 37 1 0 14 Oct 2024
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models Fei Wang Li Shen Liang Ding Chao Xue Ye Liu Changxing Ding 28 0 0 13 Oct 2024
Bi-TTA: Bidirectional Test-Time Adapter for Remote Physiological Measurement Haodong Li Hao Lu Ying-Cong Chen 23 1 0 25 Sep 2024
Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate Hinata Harada Hideaki Iiduka 28 1 0 16 Sep 2024
Enhancing Sharpness-Aware Minimization by Learning Perturbation Radius Xuehao Wang Weisen Jiang Shuai Fu Yu Zhang AAML 39 0 0 15 Aug 2024
Flat Posterior Does Matter For Bayesian Model Averaging Sungjun Lim Jeyoon Yeom Sooyon Kim Hoyoon Byun Jinho Kang Yohan Jung Jiyoung Jung Kyungwoo Song AAML BDL 43 0 0 21 Jun 2024
A Universal Class of Sharpness-Aware Minimization Algorithms B. Tahmasebi Ashkan Soleymani Dara Bahri Stefanie Jegelka P. Jaillet AAML 47 2 0 06 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 37 6 0 31 May 2024
Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization Ziqing Fan Shengchao Hu Jiangchao Yao Gang Niu Ya-Qin Zhang Masashi Sugiyama Yanfeng Wang FedML 39 11 0 29 May 2024
Domain-Inspired Sharpness-Aware Minimization Under Domain Shifts Ruipeng Zhang Ziqing Fan Jiangchao Yao Ya-Qin Zhang Yanfeng Wang 36 7 0 29 May 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 37 0 0 22 May 2024
Revisiting Random Weight Perturbation for Efficiently Improving Generalization Tao Li Qinghua Tao Weihao Yan Zehao Lei Yingwen Wu Kun Fang M. He Xiaolin Huang AAML 24 5 0 30 Mar 2024
Friendly Sharpness-Aware Minimization Tao Li Pan Zhou Zhengbao He Xinwen Cheng Xiaolin Huang AAML 46 15 0 19 Mar 2024
On the Duality Between Sharpness-Aware Minimization and Adversarial Training Yihao Zhang Hangzhou He Jingyu Zhu Huanran Chen Yifei Wang Zeming Wei AAML 29 9 0 23 Feb 2024
Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima Shan Zhong Zhongzhan Huang Daifeng Li Wushao Wen Jinghui Qin Liang Lin 16 12 0 17 Feb 2024
Vaccine: Perturbation-aware Alignment for Large Language Model Tiansheng Huang Sihao Hu Ling Liu 44 33 0 02 Feb 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 74 5 0 22 Jan 2024
Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy Chengli Tan Jiangshe Zhang Junmin Liu Yicheng Wang Yunda Hao AAML 26 1 0 14 Jan 2024
CR-SAM: Curvature Regularized Sharpness-Aware Minimization Tao Wu Tie Luo D. C. Wunsch 8 3 0 21 Dec 2023
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 39 1 0 29 Nov 2023
Asymmetrically Decentralized Federated Learning Qinglun Li Miao Zhang Nan Yin Quanjun Yin Li Shen FedML 22 4 0 08 Oct 2023
Enhancing Sharpness-Aware Optimization Through Variance Suppression Bingcong Li G. Giannakis AAML 21 19 0 27 Sep 2023
FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning Gihun Lee Minchan Jeong Sangmook Kim Jaehoon Oh Se-Young Yun FedML 19 8 0 24 Aug 2023
Flatness-Aware Minimization for Domain Generalization Xingxuan Zhang Renzhe Xu Han Yu Yancheng Dong Pengfei Tian Peng Cu 14 20 0 20 Jul 2023
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer Peng Mi Li Shen Tianhe Ren Yiyi Zhou Tianshuo Xu Xiaoshuai Sun Tongliang Liu Rongrong Ji Dacheng Tao AAML 26 2 0 30 Jun 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
Normalization Layers Are All That Sharpness-Aware Minimization Needs Maximilian Mueller Tiffany J. Vlaar David Rolnick Matthias Hein 8 18 0 07 Jun 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 34 15 0 05 Jun 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 21 17 0 24 May 2023
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training Yi Shi Yingqi Liu Yan Sun Zihao Lin Li Shen Xueqian Wang Dacheng Tao FedML 31 10 0 24 May 2023
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape Yan Sun Li Shen Shi-Yong Chen Liang Ding Dacheng Tao FedML 22 33 0 19 May 2023
Sharpness-Aware Minimization Alone can Improve Adversarial Robustness Zeming Wei Jingyu Zhu Yihao Zhang AAML 12 9 0 09 May 2023
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy Yi Shi Kang Wei Li Shen Yingqi Liu Xueqian Wang Bo Yuan Dacheng Tao FedML 28 2 0 01 May 2023
Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization Mingli Zhu Shaokui Wei Li Shen Yanbo Fan Baoyuan Wu AAML 31 51 0 24 Apr 2023
Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization Zhuo Huang Miaoxi Zhu Xiaobo Xia Li Shen Jun Yu Chen Gong Bo Han Bo Du Tongliang Liu 30 30 0 23 Mar 2023
Make Landscape Flatter in Differentially Private Federated Learning Yi Shi Yingqi Liu Kang Wei Li Shen Xueqian Wang Dacheng Tao FedML 17 54 0 20 Mar 2023
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization Xingxuan Zhang Renzhe Xu Han Yu Hao Zou Peng Cui 13 39 0 03 Mar 2023