PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization

24 November 2022

Sanyam Kapoor

Papers citing "PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization"

41 / 41 papers shown

Title
Compute-Optimal LLMs Provably Generalize Better With Scale Marc Finzi Sanyam Kapoor Diego Granziol Anming Gu Christopher De Sa J. Zico Kolter Andrew Gordon Wilson 28 0 0 21 Apr 2025
How good is PAC-Bayes at explaining generalisation? Antoine Picard-Weibel Eugenio Clerico Roman Moscoviz Benjamin Guedj 59 0 0 11 Mar 2025
Non-vacuous Generalization Bounds for Deep Neural Networks without any modification to the trained models Khoat Than Dat Phan BDL AAML VLM 60 0 0 10 Mar 2025
Deep Learning is Not So Mysterious or Different Andrew Gordon Wilson 36 2 0 03 Mar 2025
`Generalization is hallucination' through the lens of tensor completions Liang Ze Wong VLM 58 0 0 24 Feb 2025
On the Generalization of Preference Learning with DPO Shawn Im Yixuan Li 44 1 0 06 Aug 2024
Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models Sanae Lotfi Yilun Kuang Brandon Amos Micah Goldblum Marc Finzi Andrew Gordon Wilson 24 7 0 25 Jul 2024
Just How Flexible are Neural Networks in Practice? Ravid Shwartz-Ziv Micah Goldblum Arpit Bansal C. B. Bruss Yann LeCun Andrew Gordon Wilson 35 4 0 17 Jun 2024
Slicing Mutual Information Generalization Bounds for Neural Networks Kimia Nadjahi Kristjan Greenewald Rickard Brüel-Gabrielsson Justin Solomon 41 3 0 06 Jun 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 36 12 0 03 May 2024
How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers G. Buzaglo I. Harel Mor Shpigel Nacson Alon Brutzkus Nathan Srebro Daniel Soudry 54 3 0 09 Feb 2024
Multitask methods for predicting molecular properties from heterogeneous data Katharine Fisher Michael Herbst Youssef Marzouk 15 6 0 31 Jan 2024
Non-Vacuous Generalization Bounds for Large Language Models Sanae Lotfi Marc Finzi Yilun Kuang Tim G. J. Rudner Micah Goldblum Andrew Gordon Wilson 21 20 0 28 Dec 2023
A note on regularised NTK dynamics with an application to PAC-Bayesian training Eugenio Clerico Benjamin Guedj 33 0 0 20 Dec 2023
Perspectives on the State and Future of Deep Learning - 2023 Micah Goldblum A. Anandkumar Richard Baraniuk Tom Goldstein Kyunghyun Cho Zachary Chase Lipton Melanie Mitchell Preetum Nakkiran Max Welling Andrew Gordon Wilson 53 4 0 07 Dec 2023
PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction Apoorva Sharma Sushant Veer Asher Hancock Heng Yang Marco Pavone Anirudha Majumdar 41 8 0 07 Dec 2023
Simplifying Neural Network Training Under Class Imbalance Ravid Shwartz-Ziv Micah Goldblum Yucen Lily Li C. B. Bruss Andrew Gordon Wilson 26 14 0 05 Dec 2023
A PAC-Bayesian Perspective on the Interpolating Information Criterion Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 27 1 0 13 Nov 2023
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 55 1 0 08 Nov 2023
Estimating optimal PAC-Bayes bounds with Hamiltonian Monte Carlo Szilvia Ujváry Gergely Flamich Vincent Fortuin José Miguel Hernández Lobato 10 0 0 30 Oct 2023
Comparing Comparators in Generalization Bounds Fredrik Hellström Benjamin Guedj 23 4 0 16 Oct 2023
Understanding prompt engineering may not require rethinking generalization Victor Akinwande Yiding Jiang Dylan Sam J. Zico Kolter VLM VPVLM 115 7 0 06 Oct 2023
The fine print on tempered posteriors Konstantinos Pitas Julyan Arbel 25 1 0 11 Sep 2023
Input margins can predict generalization too Coenraad Mouton Marthinus W. Theunissen Marelie Hattingh Davel AAML UQCV AI4CE 16 3 0 29 Aug 2023
More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validity Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund 26 9 0 21 Jun 2023
Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often! Milad Sefidgaran Romain Chor A. Zaidi Yijun Wan FedML 21 6 0 09 Jun 2023
Learning via Wasserstein-Based High Probability Generalisation Bounds Paul Viallard Maxime Haddouche Umut Simsekli Benjamin Guedj 30 12 0 07 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 28 9 0 01 Jun 2023
How Does Information Bottleneck Help Deep Learning? Kenji Kawaguchi Zhun Deng Xu Ji Jiaoyang Huang 38 53 0 30 May 2023
Fundamental Tradeoffs in Learning with Prior Information Anirudha Majumdar 13 0 0 26 Apr 2023
The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning Micah Goldblum Marc Finzi K. Rowan A. Wilson UQCV FedML 13 37 0 11 Apr 2023
Mathematical Challenges in Deep Learning V. Nia Guojun Zhang I. Kobyzev Michael R. Metel Xinlin Li ... S. Hemati M. Asgharian Linglong Kong Wulong Liu Boxing Chen AI4CE VLM 35 1 0 24 Mar 2023
Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks Andong Wang C. Li Mingyuan Bai Zhong Jin Guoxu Zhou Qianchuan Zhao OOD AAML 13 5 0 01 Mar 2023
Norm-based Generalization Bounds for Compositionally Sparse Neural Networks Tomer Galanti Mengjia Xu Liane Galanti T. Poggio 14 9 0 28 Jan 2023
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 8 10 0 19 Nov 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 33 1 0 09 Jun 2022
Predicting the generalization gap in neural networks using topological data analysis Rubén Ballester Xavier Arnal Clemente Carles Casacuberta Meysam Madadi C. Corneanu Sergio Escalera 33 2 0 23 Mar 2022
Bayesian Model Selection, the Marginal Likelihood, and Generalization Sanae Lotfi Pavel Izmailov Gregory W. Benton Micah Goldblum A. Wilson UQCV BDL 52 56 0 23 Feb 2022
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 81 72 0 29 Sep 2021
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,886 0 15 Sep 2016
Pac-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning O. Catoni 137 453 0 03 Dec 2007