v1v2v3 (latest)

Information-Theoretic Generalization Bounds for Stochastic Gradient Descent

1 February 2021

Gergely Neu

Gintare Karolina Dziugaite

Mahdi Haghifam

Daniel M. Roy

ArXiv (abs)PDF HTML

Papers citing "Information-Theoretic Generalization Bounds for Stochastic Gradient Descent"

50 / 71 papers shown

Title
Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel Yilan Chen Zhichao Wang Wei Huang Andi Han Taiji Suzuki Arya Mazumdar MLT 20 0 0 12 Jun 2025
Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective Firas Laakom Haobo Chen Jürgen Schmidhuber Yuheng Bu 14 0 0 09 Jun 2025
Algorithm- and Data-Dependent Generalization Bounds for Score-Based Generative Models Benjamin Dupuis Dario Shariatian Maxime Haddouche Alain Durmus Umut Simsekli 56 0 0 04 Jun 2025
Information-theoretic Generalization Analysis for VQ-VAEs: A Role of Latent Variables Futoshi Futami Masahiro Fujisawa DRL CML 79 0 0 26 May 2025
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Romain Chor Abdellatif Zaidi Piotr Krasnowski 95 1 0 25 Apr 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 282 2 0 21 Feb 2025
Stability-based Generalization Bounds for Variational Inference Yadi Wei Roni Khardon BDL 81 0 0 17 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 101 0 0 11 Feb 2025
Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimization Dun Zeng Zheshun Wu Shiyu Liu Yu Pan Xiaoying Tang Zenglin Xu MLT FedML 166 1 0 25 Nov 2024
Generalization Bounds via Conditional $f$ -Information Ziqiao Wang Yongyi Mao FedML 139 1 0 30 Oct 2024
Bootstrap SGD: Algorithmic Stability and Robustness Andreas Christmann Yunwen Lei 55 0 0 02 Sep 2024
Deep Companion Learning: Enhancing Generalization Through Historical Consistency Ruizhao Zhu Venkatesh Saligrama FedML 87 0 0 26 Jul 2024
How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis Yuxin Dong Tieliang Gong Hong Chen Shuangyong Song Weizhan Zhang Chen Li OOD 85 1 0 14 Jun 2024
Enhancing Domain Adaptation through Prompt Gradient Alignment Hoang Phan Lam C. Tran Quyen Tran Trung Le 173 1 0 13 Jun 2024
Error Bounds of Supervised Classification from Information-Theoretic Perspective Binchuan Qi Wei Gong Li Li 58 0 0 07 Jun 2024
Finite Sample Analysis and Bounds of Generalization Error of Gradient Descent in In-Context Linear Regression Karthik Duraisamy MLT 85 3 0 03 May 2024
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets Benjamin Dupuis Paul Viallard George Deligiannidis Umut Simsekli 129 5 0 26 Apr 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 108 6 0 04 Apr 2024
A PAC-Bayesian Link Between Generalisation and Flat Minima Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj 116 4 0 13 Feb 2024
Enhancing selectivity using Wasserstein distance based reweighing Pratik Worah OOD 114 0 0 21 Jan 2024
Convex SGD: Generalization Without Early Stopping Julien Hendrickx A. Olshevsky MLT LRM 62 1 0 08 Jan 2024
Class-wise Generalization Error: an Information-Theoretic Analysis Firas Laakom Yuheng Bu Moncef Gabbouj 76 1 0 05 Jan 2024
A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure Yanjie Li Weijun Li Lina Yu Min Wu Jinyi Liu ... Xin Ning Yugui Zhang Baoli Lu Jian Xu Shuang Li 74 0 0 03 Jan 2024
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 152 0 0 08 Nov 2023
Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds Ziqiao Wang Yongyi Mao 87 7 0 31 Oct 2023
Information-Theoretic Generalization Analysis for Topology-aware Heterogeneous Federated Edge Learning over Noisy Channels Zheshun Wu Zenglin Xu Hongfang Yu Jie Liu 65 5 0 25 Oct 2023
Sample-Driven Federated Learning for Energy-Efficient and Real-Time IoT Sensing Minh Ngoc Luu Minh-Duong Nguyen E. Bedeer Van Duc Nguyen D. Hoang Diep N. Nguyen Quoc-Viet Pham 62 3 0 11 Oct 2023
Generalization error bounds for iterative learning algorithms with bounded updates Jingwen Fu Nanning Zheng 79 1 0 10 Sep 2023
Minimax Excess Risk of First-Order Methods for Statistical Learning with Data-Dependent Oracles Kevin Scaman Mathieu Even B. L. Bars Laurent Massoulié 44 1 0 10 Jul 2023
Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm B. L. Bars A. Bellet Marc Tommasi Kevin Scaman Giovanni Neglia 58 2 0 05 Jun 2023
Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training Rie Johnson Tong Zhang 43 6 0 31 May 2023
Online-to-PAC Conversions: Generalization Bounds via Regret Analysis Gábor Lugosi Gergely Neu 85 12 0 31 May 2023
On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequences A. F. Pour H. Ashtiani 73 0 0 28 May 2023
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective Yuxin Dong Tieliang Gong Hao Chen Chen Li 60 3 0 02 May 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 86 3 0 25 Apr 2023
More Communication Does Not Result in Smaller Generalization Error in Federated Learning Abdellatif Zaidi Romain Chor Milad Sefidgaran FedML AI4CE 99 10 0 24 Apr 2023
Generalization Error Bounds for Noisy, Iterative Algorithms via Maximal Leakage Ibrahim Issa A. Esposito Michael C. Gastpar 56 2 0 28 Feb 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 112 16 0 09 Feb 2023
Tighter Information-Theoretic Generalization Bounds from Supersamples Ziqiao Wang Yongyi Mao 109 19 0 05 Feb 2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions Anant Raj Lingjiong Zhu Mert Gurbuzbalaban Umut Simsekli 78 16 0 27 Jan 2023
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 89 19 0 27 Dec 2022
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 111 12 0 19 Nov 2022
A New Family of Generalization Bounds Using Samplewise Evaluated CMI Fredrik Hellström G. Durisi 92 26 0 12 Oct 2022
Information-Theoretic Analysis of Unsupervised Domain Adaptation Ziqiao Wang Yongyi Mao 119 11 0 03 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 379 50 0 29 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 152 4 0 06 Sep 2022
Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality Wave Ngampruetikorn David J. Schwab 59 7 0 08 Aug 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 95 6 0 01 Jul 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 66 3 0 14 Jun 2022
Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems Yunwen Lei 37 19 0 14 Jun 2022