Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data

31 March 2017

Papers citing "Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data"

50 / 168 papers shown

Title
In Search of Probeable Generalization Measures Jonathan Jaegerman Khalil Damouni M. M. Ankaralı Konstantinos N. Plataniotis 19 2 0 23 Oct 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 20 2 0 22 Oct 2021
User-friendly introduction to PAC-Bayes bounds Pierre Alquier FedML 48 196 0 21 Oct 2021
VC dimension of partially quantized neural networks in the overparametrized regime Yutong Wang Clayton D. Scott 14 1 0 06 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 38 5 0 01 Oct 2021
Ridgeless Interpolation with Shallow ReLU Networks in $1D$ is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions Boris Hanin MLT 32 9 0 27 Sep 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 29 71 0 06 Sep 2021
The Bayesian Learning Rule Mohammad Emtiyaz Khan Håvard Rue BDL 55 73 0 09 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 26 17 0 04 Jul 2021
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes Peter Grünwald Thomas Steinke Lydia Zakynthinou 21 29 0 17 Jun 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 25 29 0 09 Jun 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 28 29 0 01 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 31 13 0 29 Apr 2021
Generalization bounds via distillation Daniel J. Hsu Ziwei Ji Matus Telgarsky Lan Wang FedML 13 32 0 12 Apr 2021
Risk Bounds for Learning via Hilbert Coresets Spencer Douglas Piyush Kumar R. Prasanth 16 0 0 29 Mar 2021
PAC-Bayesian theory for stochastic LTI systems Deividas Eringis J. Leth Z. Tan Rafal Wisniewski Alireza Fakhrizadeh Esfahani M. Petreczky 11 9 0 23 Mar 2021
Computing the Information Content of Trained Neural Networks Jeremy Bernstein Yisong Yue 14 4 0 01 Mar 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 13 14 0 21 Feb 2021
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability Alec Farid Anirudha Majumdar 24 34 0 12 Feb 2021
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 16 55 0 14 Dec 2020
On 1/n neural representation and robustness Josue Nassar Piotr A. Sokól SueYeon Chung K. Harris Il Memming Park AAML OOD 16 23 0 08 Dec 2020
Margin-Based Transfer Bounds for Meta Learning with Deep Feature Embedding Jiechao Guan Zhiwu Lu Tao Xiang Timothy M. Hospedales 16 0 0 02 Dec 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 34 23 0 27 Oct 2020
For self-supervised learning, Rationality implies generalization, provably Yamini Bansal Gal Kaplun Boaz Barak OOD SSL 58 22 0 16 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 31 1,276 0 03 Oct 2020
Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary Shuyue Guan Murray H. Loew 17 25 0 16 Sep 2020
PAC-Bayes Analysis Beyond the Usual Bounds Omar Rivasplata Ilja Kuzborskij Csaba Szepesvári John Shawe-Taylor 27 80 0 23 Jun 2020
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks Umut Simsekli Ozan Sener George Deligiannidis Murat A. Erdogdu 41 55 0 16 Jun 2020
Speedy Performance Estimation for Neural Architecture Search Binxin Ru Clare Lyle Lisa Schut M. Fil Mark van der Wilk Y. Gal 15 35 0 08 Jun 2020
Classification vs regression in overparameterized regimes: Does the loss function matter? Vidya Muthukumar Adhyyan Narang Vignesh Subramanian M. Belkin Daniel J. Hsu A. Sahai 36 148 0 16 May 2020
Generalization Bounds via Information Density and Conditional Information Density Fredrik Hellström G. Durisi 15 65 0 16 May 2020
Probably Approximately Correct Vision-Based Planning using Motion Primitives Sushant Veer Anirudha Majumdar 3DV 14 21 0 28 Feb 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization A. Wilson Pavel Izmailov UQCV BDL OOD 19 639 0 20 Feb 2020
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima Zeke Xie Issei Sato Masashi Sugiyama ODL 20 17 0 10 Feb 2020
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin Colin Wei Tengyu Ma AAML OOD 30 85 0 09 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 11 116 0 03 Oct 2019
Minimum Description Length Revisited Peter Grünwald Teemu Roos 15 64 0 21 Aug 2019
PAC-Bayes with Backprop Omar Rivasplata Vikram Tankasali Csaba Szepesvári 18 49 0 19 Aug 2019
A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems Meng Tang Yimin Liu L. Durlofsky AI4CE 19 255 0 16 Aug 2019
On improving deep learning generalization with adaptive sparse connectivity Shiwei Liu D. Mocanu Mykola Pechenizkiy ODL 12 7 0 27 Jun 2019
Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets Amir-Reza Asadi Emmanuel Abbe BDL AI4CE 31 13 0 26 Jun 2019
Learning to Forget for Meta-Learning Sungyong Baik Seokil Hong Kyoung Mu Lee CLL KELM 14 87 0 13 Jun 2019
Stochastic Neural Network with Kronecker Flow Chin-Wei Huang Ahmed Touati Pascal Vincent Gintare Karolina Dziugaite Alexandre Lacoste Aaron Courville BDL 24 8 0 10 Jun 2019
PAC-Bayes Un-Expected Bernstein Inequality Zakaria Mhammedi Peter Grünwald Benjamin Guedj 16 46 0 31 May 2019
Generalization bounds for deep convolutional neural networks Philip M. Long Hanie Sedghi MLT 37 89 0 29 May 2019
Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks Gaël Letarte Pascal Germain Benjamin Guedj Franccois Laviolette MQ AI4CE UQCV 20 54 0 24 May 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 35 961 0 24 Jan 2019
Towards Theoretical Understanding of Large Batch Training in Stochastic Gradient Descent Xiaowu Dai Yuhua Zhu 17 11 0 03 Dec 2018
Neural Lander: Stable Drone Landing Control using Learned Dynamics Guanya Shi Xichen Shi Michael O'Connell Rose Yu Kamyar Azizzadenesheli Anima Anandkumar Yisong Yue Soon-Jo Chung 16 269 0 19 Nov 2018
Scalable agent alignment via reward modeling: a research direction Jan Leike David M. Krueger Tom Everitt Miljan Martic Vishal Maini Shane Legg 28 392 0 19 Nov 2018