Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms

27 April 2020

Mahdi Haghifam

Jeffrey Negrea

Ashish Khisti

Daniel M. Roy

Gintare Karolina Dziugaite

ArXiv PDF HTML

Papers citing "Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms"

29 / 29 papers shown

Title
Generalization in Federated Learning: A Conditional Mutual Information Framework Ziqiao Wang Cheng Long Yongyi Mao FedML 52 0 0 06 Mar 2025
Stability-based Generalization Bounds for Variational Inference Yadi Wei R. Khardon BDL 46 0 0 17 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 47 0 0 11 Feb 2025
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets Benjamin Dupuis Paul Viallard George Deligiannidis Umut Simsekli 40 2 0 26 Apr 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 35 4 0 04 Apr 2024
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 57 1 0 08 Nov 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 28 3 0 25 Apr 2023
Tighter Information-Theoretic Generalization Bounds from Supersamples Ziqiao Wang Yongyi Mao 29 17 0 05 Feb 2023
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 27 10 0 19 Nov 2022
Information-Theoretic Analysis of Unsupervised Domain Adaptation Ziqiao Wang Yongyi Mao 38 11 0 03 Oct 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 28 5 0 01 Jul 2022
Understanding Generalization via Leave-One-Out Conditional Mutual Information Mahdi Haghifam Shay Moran Daniel M. Roy Gintare Karolina Dziugaite 20 14 0 29 Jun 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 24 3 0 14 Jun 2022
Fast Rate Generalization Error Bounds: Variations on a Theme Xuetong Wu J. Manton U. Aickelin Jingge Zhu 22 7 0 06 May 2022
An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift Gholamali Aminian Mahed Abroshan Mohammad Mahdi Khalili Laura Toni M. Rodrigues OOD 23 27 0 24 Feb 2022
Information-Theoretic Analysis of Minimax Excess Risk Hassan Hafez-Kolahi Behrad Moniri S. Kasaei 17 4 0 15 Feb 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 31 8 0 09 Jan 2022
Towards a Unified Information-Theoretic Framework for Generalization Mahdi Haghifam Gintare Karolina Dziugaite Shay Moran Daniel M. Roy 19 34 0 09 Nov 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 34 2 0 22 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 37 22 0 07 Oct 2021
Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning Haiyun He Hanshu Yan Vincent Y. F. Tan 31 11 0 03 Oct 2021
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information Gholamali Aminian Yuheng Bu Laura Toni M. Rodrigues G. Wornell 24 4 0 28 Jul 2021
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes Peter Grünwald Thomas Steinke Lydia Zakynthinou 28 29 0 17 Jun 2021
Minimum Excess Risk in Bayesian Learning Aolin Xu Maxim Raginsky 31 37 0 29 Dec 2020
Individually Conditional Individual Mutual Information Bound on Generalization Error Ruida Zhou C. Tian Tie Liu 13 31 0 17 Dec 2020
On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm Borja Rodríguez Gálvez Germán Bassi Ragnar Thobaben Mikael Skoglund 20 32 0 21 Oct 2020
Generalization Bounds via Information Density and Conditional Information Density Fredrik Hellström G. Durisi 27 65 0 16 May 2020
Reasoning About Generalization via Conditional Mutual Information Thomas Steinke Lydia Zakynthinou 23 159 0 24 Jan 2020
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 110 147 0 06 Nov 2019