Sharp Minima Can Generalize For Deep Nets

15 March 2017

Papers citing "Sharp Minima Can Generalize For Deep Nets"

50 / 132 papers shown

Title
Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning Lianbo Ma Jianlun Ma Yuee Zhou Guoyang Xie Qiang He Zhichao Lu MQ 36 0 0 08 May 2025
Non-identifiability distinguishes Neural Networks among Parametric Models Sourav Chatterjee Timothy Sudijono 18 0 0 25 Apr 2025
High-entropy Advantage in Neural Networks' Generalizability Entao Yang X. Zhang Yue Shang Ge Zhang AI4CE 58 0 0 17 Mar 2025
Evidence on the Regularisation Properties of Maximum-Entropy Reinforcement Learning Rémy Hosseinkhan Boucher Onofrio Semeraro L. Mathelin 72 0 0 28 Jan 2025
Meta Curvature-Aware Minimization for Domain Generalization Z. Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 159 1 0 16 Dec 2024
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurélien Lucchi AI4CE 39 0 0 04 Nov 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 26 1 0 08 Oct 2024
Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes Nikita Kiselev Andrey Grabovoy 41 1 0 18 Sep 2024
Layerwise Change of Knowledge in Neural Networks Xu Cheng Lei Cheng Zhaoran Peng Yang Xu Tian Han Quanshi Zhang KELM FAtt 33 6 0 13 Sep 2024
Remove Symmetries to Control Model Expressivity and Improve Optimization Liu Ziyin Yizhou Xu Isaac Chuang AAML 38 1 0 28 Aug 2024
What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions? Weijie Tu Weijian Deng Liang Zheng Tom Gedeon 32 0 0 14 Jun 2024
Agnostic Sharpness-Aware Minimization Van-Anh Nguyen Quyen Tran Tuan Truong Thanh-Toan Do Dinh Q. Phung Trung Le 38 0 0 11 Jun 2024
Reparameterization invariance in approximate Bayesian inference Hrittik Roy M. Miani Carl Henrik Ek Philipp Hennig Marvin Pfortner Lukas Tatzel Søren Hauberg BDL 42 8 0 05 Jun 2024
The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective Nils Philipp Walter Linara Adilova Jilles Vreeken Michael Kamp AAML 43 2 0 27 May 2024
A separability-based approach to quantifying generalization: which layer is best? Luciano Dyballa Evan Gerritz Steven W. Zucker OOD 24 3 0 02 May 2024
Flatness Improves Backbone Generalisation in Few-shot Classification Rui Li Martin Trapp Marcus Klasson Arno Solin 41 0 0 11 Apr 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 35 4 0 04 Apr 2024
Level Set Teleportation: An Optimization Perspective Aaron Mishkin A. Bietti Robert Mansel Gower 28 1 0 05 Mar 2024
Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures Paul Viallard Rémi Emonet Amaury Habrard Emilie Morvant Valentina Zantedeschi 31 3 0 19 Feb 2024
AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size P. Ostroukhov Aigerim Zhumabayeva Chulu Xiang Alexander Gasnikov Martin Takáč Dmitry Kamzolov ODL 38 2 0 07 Feb 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 74 5 0 22 Jan 2024
Graph Metanetworks for Processing Diverse Neural Architectures Derek Lim Haggai Maron Marc T. Law Jonathan Lorraine James Lucas AI4CE 29 30 0 07 Dec 2023
Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives Pierre Wolinski ODL 24 0 0 06 Dec 2023
Lookbehind-SAM: k steps back, 1 step forward Gonçalo Mordido Pranshu Malviya A. Baratin Sarath Chandar AAML 38 1 0 31 Jul 2023
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization Kaiyue Wen Zhiyuan Li Tengyu Ma FAtt 22 26 0 20 Jul 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 22 6 0 25 May 2023
A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks Marc Finzi Andres Potapczynski M. Choptuik A. Wilson 13 12 0 28 Apr 2023
Mathematical Challenges in Deep Learning V. Nia Guojun Zhang I. Kobyzev Michael R. Metel Xinlin Li ... S. Hemati M. Asgharian Linglong Kong Wulong Liu Boxing Chen AI4CE VLM 35 1 0 24 Mar 2023
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent Liu Ziyin Botao Li Tomer Galanti Masakuni Ueda 37 7 0 23 Mar 2023
Randomized Adversarial Training via Taylor Expansion Gao Jin Xinping Yi Dengyu Wu Ronghui Mu Xiaowei Huang AAML 31 34 0 19 Mar 2023
ASP: Learn a Universal Neural Solver! Chenguang Wang Zhouliang Yu Stephen Marcus McAleer Tianshu Yu Yao-Chun Yang AAML 30 23 0 01 Mar 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 13 7 0 19 Feb 2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon Atish Agarwala Yann N. Dauphin 17 20 0 17 Feb 2023
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 22 11 0 14 Feb 2023
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning Antonio Sclocchi Mario Geiger M. Wyart 32 6 0 31 Jan 2023
Exploring the Effect of Multi-step Ascent in Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Woojin Lee Jaewook Lee 13 9 0 27 Jan 2023
Catapult Dynamics and Phase Transitions in Quadratic Nets David Meltzer Junyu Liu 15 9 0 18 Jan 2023
A survey of deep learning optimizers -- first and second order methods Rohan Kashyap ODL 16 6 0 28 Nov 2022
The Vanishing Decision Boundary Complexity and the Strong First Component Hengshuai Yao UQCV 21 0 0 25 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 21 47 0 10 Nov 2022
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization Xiang Li Junchi Yang Niao He 24 8 0 31 Oct 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo-Lu Zhao I. Ganev Robin G. Walters Rose Yu Nima Dehmamy 42 16 0 31 Oct 2022
Sufficient Invariant Learning for Distribution Shift Taero Kim Sungjun Lim Kyungwoo Song OOD 19 2 0 24 Oct 2022
Rethinking Sharpness-Aware Minimization as Variational Inference Szilvia Ujváry Zsigmond Telek A. Kerekes Anna Mészáros Ferenc Huszár 17 8 0 19 Oct 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models Nikolaos Dimitriadis P. Frossard Franccois Fleuret 16 25 0 18 Oct 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 27 69 0 11 Oct 2022
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 63 34 0 04 Oct 2022
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel Sungyub Kim Si-hun Park Kyungsu Kim Eunho Yang BDL 24 4 0 30 Sep 2022
Deep Double Descent via Smooth Interpolation Matteo Gamba Erik Englesson Marten Bjorkman Hossein Azizpour 51 10 0 21 Sep 2022