v1v2 (latest)

Tightening Mutual Information Based Bounds on Generalization Error

15 January 2019

Yuheng Bu

Shaofeng Zou

Venugopal V. Veeravalli

ArXiv (abs)PDF HTML

Papers citing "Tightening Mutual Information Based Bounds on Generalization Error"

50 / 118 papers shown

Title
Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective Firas Laakom Haobo Chen Jürgen Schmidhuber Yuheng Bu 14 0 0 09 Jun 2025
Trade-offs in Data Memorization via Strong Data Processing Inequalities Vitaly Feldman Guy Kornowski Xin Lyu TDI FedML 62 0 0 02 Jun 2025
Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis Qi Chen Jierui Zhu Florian Shkurti DiffM 65 1 0 01 Jun 2025
Bounds on the Excess Minimum Risk via Generalized Information Divergence Measures Ananya Omanwar F. Alajaji Tamás Linder 35 0 0 30 May 2025
Generalization Bounds for Quantum Learning via Rényi Divergences Naqueeb Ahmad Warsi Ayanava Dasgupta Masahito Hayashi 53 0 0 16 May 2025
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Romain Chor Abdellatif Zaidi Piotr Krasnowski 95 1 0 25 Apr 2025
Generalization in Federated Learning: A Conditional Mutual Information Framework Ziqiao Wang Cheng Long Yongyi Mao FedML 99 1 0 06 Mar 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 285 2 0 21 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 101 0 0 11 Feb 2025
The Generalization Error of Machine Learning Algorithms S. Perlaza Xinying Zou 118 6 0 18 Nov 2024
Generalization Bounds via Conditional $f$ -Information Ziqiao Wang Yongyi Mao FedML 139 1 0 30 Oct 2024
Generalization and Robustness of the Tilted Empirical Risk Gholamali Aminian Amir R. Asadi Tian Li Ahmad Beirami Gesine Reinert Samuel N. Cohen 71 0 0 28 Sep 2024
Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization Wenrui Yu Qiongxiu Li Milan Lopuhaä-Zwakenberg Mads Græsbøll Christensen Richard Heusdens FedML 85 4 0 12 Jul 2024
How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis Yuxin Dong Tieliang Gong Hong Chen Shuangyong Song Weizhan Zhang Chen Li OOD 85 1 0 14 Jun 2024
Error Bounds of Supervised Classification from Information-Theoretic Perspective Binchuan Qi Wei Gong Li Li 58 0 0 07 Jun 2024
Slicing Mutual Information Generalization Bounds for Neural Networks Kimia Nadjahi Kristjan Greenewald Rickard Brüel-Gabrielsson Justin Solomon 99 4 0 06 Jun 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 110 6 0 04 Apr 2024
An Information-Theoretic Framework for Out-of-Distribution Generalization Wenliang Liu Guanding Yu Lele Wang Renjie Liao 70 3 0 29 Mar 2024
Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures Paul Viallard Rémi Emonet Amaury Habrard Emilie Morvant Valentina Zantedeschi 116 3 0 19 Feb 2024
Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization Idan Attias Gintare Karolina Dziugaite Mahdi Haghifam Roi Livni Daniel M. Roy 99 7 0 14 Feb 2024
Minimum Description Length and Generalization Guarantees for Representation Learning Romain Chor Abdellatif Zaidi Piotr Krasnowski 138 10 0 05 Feb 2024
No Free Prune: Information-Theoretic Barriers to Pruning at Initialization Tanishq Kumar Kevin Luo Mark Sellke 78 3 0 02 Feb 2024
Class-wise Generalization Error: an Information-Theoretic Analysis Firas Laakom Yuheng Bu Moncef Gabbouj 76 1 0 05 Jan 2024
Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measure Xinying Zou S. Perlaza I. Esnaola Eitan Altman 58 18 0 19 Dec 2023
Adaptive Differentially Quantized Subspace Perturbation (ADQSP): A Unified Framework for Privacy-Preserving Distributed Average Consensus Qiongxiu Li Jaron Skovsted Gundersen Milan Lopuhaä-Zwakenberg Richard Heusdens 69 11 0 13 Dec 2023
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 152 0 0 08 Nov 2023
Time-Independent Information-Theoretic Generalization Bounds for SGLD Futoshi Futami Masahiro Fujisawa 112 8 0 02 Nov 2023
Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds Ziqiao Wang Yongyi Mao 87 7 0 31 Oct 2023
Comparing Comparators in Generalization Bounds Fredrik Hellström Benjamin Guedj 76 4 0 16 Oct 2023
Stochastic Thermodynamics of Learning Parametric Probabilistic Models S. Parsi 81 0 0 04 Oct 2023
More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validity Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund 150 9 0 21 Jun 2023
On the Generalization of Diffusion Model Mingyang Yi Jiacheng Sun Zhenguo Li 92 21 0 24 May 2023
Feasibility of Transfer Learning: A Mathematical Framework Haoyang Cao Haotian Gu Xin Guo 75 5 0 22 May 2023
A unified framework for information-theoretic generalization bounds Y.-C. Chu Maxim Raginsky 47 16 0 18 May 2023
Generalization Bounds for Neural Belief Propagation Decoders Sudarshan Adiga Xin Xiao Ravi Tandon Bane V. Vasic Tamal Bose BDL AI4CE 78 5 0 17 May 2023
Exactly Tight Information-Theoretic Generalization Error Bound for the Quadratic Gaussian Problem Ruida Zhou C. Tian Tie Liu 80 3 0 01 May 2023
Recognizable Information Bottleneck Yilin Lyu Xin Liu M. Song Xinyue Wang Chaomin Shen T. Zeng L. Jing 117 5 0 28 Apr 2023
On the Generalization Error of Meta Learning for the Gibbs Algorithm Yuheng Bu Harsha Vardhan Tetali Gholamali Aminian Miguel R. D. Rodrigues G. Wornell AI4CE 89 3 0 27 Apr 2023
More Communication Does Not Result in Smaller Generalization Error in Federated Learning Abdellatif Zaidi Romain Chor Milad Sefidgaran FedML AI4CE 99 10 0 24 Apr 2023
Generalization Error Bounds for Noisy, Iterative Algorithms via Maximal Leakage Ibrahim Issa A. Esposito Michael C. Gastpar 56 2 0 28 Feb 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 112 16 0 09 Feb 2023
Tighter Information-Theoretic Generalization Bounds from Supersamples Ziqiao Wang Yongyi Mao 109 19 0 05 Feb 2023
Feasibility and Transferability of Transfer Learning: A Mathematical Framework Haoyang Cao Haotian Gu Xin Guo M. Rosenbaum 73 2 0 27 Jan 2023
An Entropy-Based Model for Hierarchical Learning Amir-Reza Asadi 73 1 0 30 Dec 2022
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 89 19 0 27 Dec 2022
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 111 12 0 19 Nov 2022
How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm? Haiyun He Gholamali Aminian Yuheng Bu Miguel R. D. Rodrigues Vincent Y. F. Tan 48 4 0 15 Oct 2022
Evaluated CMI Bounds for Meta Learning: Tightness and Expressiveness Fredrik Hellström G. Durisi 71 13 0 12 Oct 2022
A New Family of Generalization Bounds Using Samplewise Evaluated CMI Fredrik Hellström G. Durisi 92 26 0 12 Oct 2022
Over-the-Air Split Machine Learning in Wireless MIMO Networks YuZhi Yang Zhaoyang Zhang Yuqing Tian Zhaohui Yang Chongwen Huang C. Zhong Kai‐Kit Wong 72 23 0 07 Oct 2022