Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data

31 March 2017

Papers citing "Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data"

50 / 168 papers shown

Title
Mixed moving average field guided learning for spatio-temporal data I. Curato O. Furat Lorenzo Proietti Bennet Stroeh AI4TS 23 2 0 02 Jan 2023
PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models Deividas Eringis J. Leth Z. Tan Rafal Wisniewski M. Petreczky 27 1 0 30 Dec 2022
Adaptive adversarial training method for improving multi-scale GAN based on generalization bound theory Jin-Lin Tang B. Tao Zeyu Gong Zhoupin Yin AI4CE 28 1 0 30 Nov 2022
PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization Sanae Lotfi Marc Finzi Sanyam Kapoor Andres Potapczynski Micah Goldblum A. Wilson BDL MLT AI4CE 24 51 0 24 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 23 47 0 10 Nov 2022
Instance-Dependent Generalization Bounds via Optimal Transport Songyan Hou Parnian Kassraie Anastasis Kratsios Andreas Krause Jonas Rothfuss 20 6 0 02 Nov 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 34 49 0 25 Oct 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 43 8 0 25 Oct 2022
A PAC-Bayesian Generalization Bound for Equivariant Networks Arash Behboodi Gabriele Cesa Taco S. Cohen 48 17 0 24 Oct 2022
Rethinking Sharpness-Aware Minimization as Variational Inference Szilvia Ujváry Zsigmond Telek A. Kerekes Anna Mészáros Ferenc Huszár 25 8 0 19 Oct 2022
Optimisation & Generalisation in Networks of Neurons Jeremy Bernstein AI4CE 16 2 0 18 Oct 2022
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 70 34 0 04 Oct 2022
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel Sungyub Kim Si-hun Park Kyungsu Kim Eunho Yang BDL 26 4 0 30 Sep 2022
A Note on the Efficient Evaluation of PAC-Bayes Bounds Felix Biggs 37 1 0 12 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 26 4 0 06 Sep 2022
On Rademacher Complexity-based Generalization Bounds for Deep Learning Lan V. Truong MLT 37 13 0 08 Aug 2022
PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners Anthony Sicilia Katherine Atwell Malihe Alikhani Seong Jae Hwang BDL 51 9 0 12 Jul 2022
Integral Probability Metrics PAC-Bayes Bounds Ron Amit Baruch Epstein Shay Moran Ron Meir 21 18 0 01 Jul 2022
Cold Posteriors through PAC-Bayes Konstantinos Pitas Julyan Arbel 23 5 0 22 Jun 2022
A Closer Look at Smoothness in Domain Adversarial Training Harsh Rangwani Sumukh K Aithal Mayank Mishra Arihant Jain R. Venkatesh Babu 27 119 0 16 Jun 2022
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling Jiri Hron Roman Novak Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 42 6 0 15 Jun 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 15 3 0 14 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 35 1 0 09 Jun 2022
Online PAC-Bayes Learning Maxime Haddouche Benjamin Guedj 16 21 0 31 May 2022
Towards Size-Independent Generalization Bounds for Deep Operator Nets Pulkit Gopalani Sayar Karmakar Dibyakanti Kumar Anirbit Mukherjee AI4CE 24 5 0 23 May 2022
Investigating Generalization by Controlling Normalized Margin Alexander R. Farhang Jeremy Bernstein Kushal Tirumala Yang Liu Yisong Yue 25 6 0 08 May 2022
A PAC-Bayes oracle inequality for sparse neural networks Maximilian F. Steffen Mathias Trabs UQCV 19 2 0 26 Apr 2022
Enhancing Adversarial Training with Second-Order Statistics of Weights Gao Jin Xinping Yi Wei Huang S. Schewe Xiaowei Huang AAML 17 47 0 11 Mar 2022
PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks Nan Ding Xi Chen Tomer Levinboim Soravit Changpinyo Radu Soricut 22 26 0 10 Mar 2022
Generalization Through The Lens Of Leave-One-Out Error Gregor Bachmann Thomas Hofmann Aurélien Lucchi 44 7 0 07 Mar 2022
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 15 13 0 27 Feb 2022
Adversarial robustness of sparse local Lipschitz predictors Ramchandran Muthukumar Jeremias Sulam AAML 32 13 0 26 Feb 2022
Bayesian Model Selection, the Marginal Likelihood, and Generalization Sanae Lotfi Pavel Izmailov Gregory W. Benton Micah Goldblum A. Wilson UQCV BDL 52 56 0 23 Feb 2022
On change of measure inequalities for $f$ -divergences Antoine Picard-Weibel Benjamin Guedj 25 13 0 11 Feb 2022
The no-free-lunch theorems of supervised learning T. Sterkenburg Peter Grünwald FedML 8 55 0 09 Feb 2022
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurélien Lucchi 53 44 0 06 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Non-Vacuous Generalisation Bounds for Shallow Neural Networks Felix Biggs Benjamin Guedj BDL 30 26 0 03 Feb 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 11 58 0 01 Feb 2022
Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees Kai Hsu Allen Z. Ren D. Nguyen Anirudha Majumdar J. F. Fisac OffRL 22 41 0 20 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
Generalization Error Bounds on Deep Learning with Markov Datasets Lan V. Truong 18 8 0 23 Dec 2021
Sharpness-Aware Minimization with Dynamic Reweighting Wenxuan Zhou Fangyu Liu Huan Zhang Muhao Chen AAML 19 8 0 16 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 36 16 0 05 Dec 2021
Sharpness-aware Quantization for Deep Neural Networks Jing Liu Jianfei Cai Bohan Zhuang MQ 27 24 0 24 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 17 18 0 11 Nov 2021
Improved Regularization and Robustness for Fine-tuning in Neural Networks Dongyue Li Hongyang R. Zhang NoLa 49 54 0 08 Nov 2021
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 26 4 0 07 Nov 2021
GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily Lun Du Xiaozhou Shi Qiang Fu Xiaojun Ma Hengyu Liu Shi Han Dongmei Zhang 29 104 0 29 Oct 2021
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang J. Mao Pratik Chaudhari 22 15 0 27 Oct 2021