Fantastic Generalization Measures and Where to Find Them

4 December 2019

Papers citing "Fantastic Generalization Measures and Where to Find Them"

50 / 179 papers shown

Title
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data Yaoqing Yang Ryan Theisen Liam Hodgkinson Joseph E. Gonzalez Kannan Ramchandran Charles H. Martin Michael W. Mahoney 97 18 0 06 Feb 2022
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurelien Lucchi 81 45 0 06 Feb 2022
Non-Vacuous Generalisation Bounds for Shallow Neural Networks Felix Biggs Benjamin Guedj BDL 52 26 0 03 Feb 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 38 58 0 01 Feb 2022
Fortuitous Forgetting in Connectionist Networks Hattie Zhou Ankit Vani Hugo Larochelle Aaron Courville CLL 29 42 0 01 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 47 9 0 31 Jan 2022
Weight Expansion: A New Perspective on Dropout and Generalization Gao Jin Xinping Yi Pengfei Yang Lijun Zhang S. Schewe Xiaowei Huang 40 5 0 23 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 37 35 0 20 Jan 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Yaoyu Zhang FaML 52 66 0 19 Jan 2022
DARTS without a Validation Set: Optimizing the Marginal Likelihood M. Fil Binxin Ru Clare Lyle Y. Gal 39 2 0 24 Dec 2021
Sharpness-Aware Minimization with Dynamic Reweighting Wenxuan Zhou Fangyu Liu Huan Zhang Muhao Chen AAML 31 8 0 16 Dec 2021
Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks Ekkehard Schnoor Arash Behboodi Holger Rauhut 29 13 0 08 Dec 2021
Label-Free Model Evaluation with Semi-Structured Dataset Representations Xiaoxiao Sun Yunzhong Hou Hongdong Li Liang Zheng 32 11 0 01 Dec 2021
Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning Matías Mendieta Taojiannan Yang Pu Wang Minwoo Lee Zhengming Ding Chong Chen FedML 58 158 0 28 Nov 2021
On Predicting Generalization using GANs Yi Zhang Arushi Gupta Nikunj Saunshi Sanjeev Arora AI4CE 76 6 0 28 Nov 2021
Sharpness-aware Quantization for Deep Neural Networks Jing Liu Jianfei Cai Bohan Zhuang MQ 71 24 0 24 Nov 2021
Subspace Adversarial Training Tao Li Yingwen Wu Sizhe Chen Kun Fang Xiaolin Huang AAML OOD 59 58 0 24 Nov 2021
Depth Without the Magic: Inductive Bias of Natural Gradient Descent A. Kerekes Anna Mészáros Ferenc Huszár ODL 37 4 0 22 Nov 2021
Improved Regularization and Robustness for Fine-tuning in Neural Networks Dongyue Li Hongyang R. Zhang NoLa 55 56 0 08 Nov 2021
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 39 4 0 07 Nov 2021
Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima Guangyuan Shi Jiaxin Chen Wenlong Zhang Li-Ming Zhan Xiao-Ming Wu CLL 43 152 0 30 Oct 2021
In Search of Probeable Generalization Measures Jonathan Jaegerman Khalil Damouni M. M. Ankaralı Konstantinos N. Plataniotis 37 2 0 23 Oct 2021
Inductive Biases and Variable Creation in Self-Attention Mechanisms Benjamin L. Edelman Surbhi Goel Sham Kakade Cyril Zhang 32 119 0 19 Oct 2021
Graphs as Tools to Improve Deep Learning Methods Carlos Lassance Myriam Bontonou Mounia Hamidouche Bastien Pasdeloup Lucas Drumetz Vincent Gripon GNN AI4CE AAML 54 0 0 08 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 50 24 0 07 Oct 2021
VC dimension of partially quantized neural networks in the overparametrized regime Yutong Wang Clayton D. Scott 59 1 0 06 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 36 3 0 05 Oct 2021
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 43 26 0 01 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 61 5 0 01 Oct 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 96 73 0 29 Sep 2021
Ridgeless Interpolation with Shallow ReLU Networks in $1D$ is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions Boris Hanin MLT 52 9 0 27 Sep 2021
Shift-Curvature, SGD, and Generalization Arwen V. Bradley C. Gomez-Uribe Manish Reddy Vuyyuru 40 2 0 21 Aug 2021
Predicting with Confidence on Unseen Distributions Devin Guillory Vaishaal Shankar Sayna Ebrahimi Trevor Darrell Ludwig Schmidt UQCV OOD 30 117 0 07 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 39 17 0 04 Jul 2021
Unveiling the structure of wide flat minima in neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta Gabriele Perugini R. Zecchina 35 33 0 02 Jul 2021
Neural Network Layer Algebra: A Framework to Measure Capacity and Compression in Deep Learning Alberto Badías A. Banerjee 30 3 0 02 Jul 2021
Minimum sharpness: Scale-invariant parameter-robustness of neural networks Hikaru Ibayashi Takuo Hamaguchi Masaaki Imaizumi 30 5 0 23 Jun 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 52 157 0 17 Jun 2021
Learning distinct features helps, provably Firas Laakom Jenni Raitoharju Alexandros Iosifidis Moncef Gabbouj MLT 41 6 0 10 Jun 2021
What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments? Weijian Deng Stephen Gould Liang Zheng 39 63 0 10 Jun 2021
Encoding-dependent generalization bounds for parametrized quantum circuits Matthias C. Caro Elies Gil-Fuster Johannes Jakob Meyer Jens Eisert R. Sweke UQCV 26 101 0 07 Jun 2021
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path X. Y. Han Vardan Papyan D. Donoho AAML 41 143 0 03 Jun 2021
Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics Charles H. Martin Michael W. Mahoney 33 19 0 01 Jun 2021
Generalization bounds via distillation Daniel J. Hsu Ziwei Ji Matus Telgarsky Lan Wang FedML 39 33 0 12 Apr 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 43 65 0 09 Apr 2021
Intraclass clustering: an implicit learning ability that regularizes DNNs Simon Carbonnelle Christophe De Vleeschouwer 70 8 0 11 Mar 2021
Robustness to Pruning Predicts Generalization in Deep Neural Networks Lorenz Kuhn Clare Lyle Aidan Gomez Jonas Rothfuss Y. Gal 53 14 0 10 Mar 2021
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks Jungmin Kwon Jeongseop Kim Hyunseong Park I. Choi 55 287 0 23 Feb 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 34 14 0 21 Feb 2021
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability Alec Farid Anirudha Majumdar 34 36 0 12 Feb 2021