Sharp Minima Can Generalize For Deep Nets

15 March 2017

Papers citing "Sharp Minima Can Generalize For Deep Nets"

50 / 132 papers shown

Title
Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 24 11 0 19 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 30 72 0 26 Aug 2022
A Deep Learning Approach for the solution of Probability Density Evolution of Stochastic Systems S. Pourtakdoust Amir H. Khodabakhsh 17 12 0 05 Jul 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 67 27 0 17 Jun 2022
Efficiently Training Low-Curvature Neural Networks Suraj Srinivas Kyle Matoba Himabindu Lakkaraju F. Fleuret AAML 23 15 0 14 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 35 69 0 14 Jun 2022
Towards Understanding Sharpness-Aware Minimization Maksym Andriushchenko Nicolas Flammarion AAML 24 133 0 13 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 33 1 0 09 Jun 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 232 45 0 24 May 2022
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes Chao Ma D. Kunin Lei Wu Lexing Ying 25 27 0 24 Apr 2022
QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization Xiuying Wei Ruihao Gong Yuhang Li Xianglong Liu F. Yu MQ VLM 19 165 0 11 Mar 2022
Adversarial robustness of sparse local Lipschitz predictors Ramchandran Muthukumar Jeremias Sulam AAML 32 13 0 26 Feb 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 16 8 0 18 Feb 2022
A Geometric Understanding of Natural Gradient Qinxun Bai S. Rosenberg Wei Xu 11 2 0 13 Feb 2022
Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning Yang Zhao Hao Zhang Xiuyuan Hu 28 115 0 08 Feb 2022
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurélien Lucchi 53 44 0 06 Feb 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 11 58 0 01 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 42 8 0 31 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
Neighborhood Region Smoothing Regularization for Finding Flat Minima In Deep Neural Networks Yang Zhao Hao Zhang 19 1 0 16 Jan 2022
On Large Batch Training and Sharp Minima: A Fokker-Planck Perspective Xiaowu Dai Yuhua Zhu 12 4 0 02 Dec 2021
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 24 4 0 07 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 22 14 0 01 Nov 2021
Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction Konstantin Schurholt Dimche Kostadinov Damian Borth SSL 19 14 0 28 Oct 2021
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang J. Mao Pratik Chaudhari 15 15 0 27 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 35 5 0 01 Oct 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 28 204 0 07 Sep 2021
Logit Attenuating Weight Normalization Aman Gupta R. Ramanath Jun Shi Anika Ramachandran Sirou Zhou Mingzhou Zhou S. Keerthi 30 1 0 12 Aug 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 17 9 0 02 Aug 2021
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II Yossi Arjevani M. Field 28 18 0 21 Jul 2021
Implicit Gradient Alignment in Distributed and Federated Learning Yatin Dandi Luis Barba Martin Jaggi FedML 18 31 0 25 Jun 2021
Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators David Stutz Nandhini Chandramoorthy Matthias Hein Bernt Schiele AAML MQ 20 18 0 16 Apr 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 22 65 0 09 Apr 2021
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges Cynthia Rudin Chaofan Chen Zhi Chen Haiyang Huang Lesia Semenova Chudi Zhong FaML AI4CE LRM 43 650 0 20 Mar 2021
Is it enough to optimize CNN architectures on ImageNet? Lukas Tuggener Jürgen Schmidhuber Thilo Stadelmann 17 23 0 16 Mar 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 22 78 0 24 Feb 2021
Adversarial Training Makes Weight Loss Landscape Sharper in Logistic Regression Masanori Yamada Sekitoshi Kanai Tomoharu Iwata Tomokatsu Takahashi Yuki Yamanaka Hiroshi Takahashi Atsutoshi Kumagai AAML 8 9 0 05 Feb 2021
Chaos and Complexity from Quantum Neural Network: A study with Diffusion Metric in Machine Learning S. Choudhury Ankan Dutta Debisree Ray 17 21 0 16 Nov 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 21 2 0 15 Nov 2020
Regularizing Neural Networks via Adversarial Model Perturbation Yaowei Zheng Richong Zhang Yongyi Mao AAML 14 94 0 10 Oct 2020
The Representation Theory of Neural Networks M. Armenta Pierre-Marc Jodoin 19 30 0 23 Jul 2020
Explicit Regularisation in Gaussian Noise Injections A. Camuto M. Willetts Umut Simsekli Stephen J. Roberts Chris Holmes 8 55 0 14 Jul 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 24 32 0 18 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 27 48 0 16 Jun 2020
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them Chen Liu Mathieu Salzmann Tao R. Lin Ryota Tomioka Sabine Süsstrunk AAML 14 81 0 15 Jun 2020
AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks Majed El Helou Frederike Dumbgen Sabine Süsstrunk CLL AI4CE 22 2 0 07 Mar 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 156 233 0 04 Mar 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization A. Wilson Pavel Izmailov UQCV BDL OOD 12 639 0 20 Feb 2020
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima Zeke Xie Issei Sato Masashi Sugiyama ODL 15 17 0 10 Feb 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 30 55 0 07 Jan 2020