v1v2 (latest)

Tightening Mutual Information Based Bounds on Generalization Error

15 January 2019

Yuheng Bu

Shaofeng Zou

Venugopal V. Veeravalli

ArXiv (abs)PDF HTML

Papers citing "Tightening Mutual Information Based Bounds on Generalization Error"

50 / 118 papers shown

Title
Information-Theoretic Analysis of Unsupervised Domain Adaptation Ziqiao Wang Yongyi Mao 119 11 0 03 Oct 2022
Learning Algorithm Generalization Error Bounds via Auxiliary Distributions Gholamali Aminian Saeed Masiha Laura Toni M. Rodrigues 53 9 0 02 Oct 2022
Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality Wave Ngampruetikorn David J. Schwab 59 7 0 08 Aug 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 95 6 0 01 Jul 2022
Understanding Generalization via Leave-One-Out Conditional Mutual Information Mahdi Haghifam Shay Moran Daniel M. Roy Gintare Karolina Dziugaite 75 15 0 29 Jun 2022
Finite Littlestone Dimension Implies Finite Information Complexity Aditya Pradeep Ido Nachum Michael C. Gastpar 59 8 0 27 Jun 2022
f-divergences and their applications in lossy compression and bounding generalization error Saeed Masiha A. Gohari Mohammad Hossein Yassaee 76 14 0 21 Jun 2022
A General framework for PAC-Bayes Bounds for Meta-Learning A. Rezazadeh AI4CE 83 4 0 11 Jun 2022
Generalization Error Bounds for Deep Neural Networks Trained by SGD Mingze Wang Chao Ma 35 14 0 07 Jun 2022
Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning Romain Chor Abdellatif Zaidi Milad Sefidgaran FedML 83 15 0 06 Jun 2022
Generalization Bounds for Gradient Methods via Discrete and Continuous Prior Jun Yu Li Xu Luo Jian Li 80 4 0 27 May 2022
Formal limitations of sample-wise information-theoretic generalization bounds Hrayr Harutyunyan Greg Ver Steeg Aram Galstyan 48 2 0 13 May 2022
Fast Rate Generalization Error Bounds: Variations on a Theme Xuetong Wu J. Manton U. Aickelin Jingge Zhu 66 7 0 06 May 2022
What You See is What You Get: Principled Deep Learning via Distributional Generalization B. Kulynych Yao-Yuan Yang Yaodong Yu Jarosław Błasiok Preetum Nakkiran OOD 78 10 0 07 Apr 2022
Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms Romain Chor A. Gohari Gaël Richard Umut Simsekli 107 24 0 04 Mar 2022
Chained Generalisation Bounds Eugenio Clerico Amitis Shidani George Deligiannidis Arnaud Doucet AI4CE FedML 74 13 0 02 Mar 2022
Tighter Expected Generalization Error Bounds via Convexity of Information Measures Gholamali Aminian Yuheng Bu G. Wornell Miguel R. D. Rodrigues 78 24 0 24 Feb 2022
An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift Gholamali Aminian Mahed Abroshan Mohammad Mahdi Khalili Laura Toni M. Rodrigues OOD 114 28 0 24 Feb 2022
Single Trajectory Nonparametric Learning of Nonlinear Dynamics Ingvar M. Ziemann H. Sandberg Nikolai Matni 71 25 0 16 Feb 2022
Information-Theoretic Analysis of Minimax Excess Risk Hassan Hafez-Kolahi Behrad Moniri S. Kasaei 54 5 0 15 Feb 2022
Black-Box Generalization: Stability of Zeroth-Order Learning Konstantinos E. Nikolakakis Farzin Haddadpour Dionysios S. Kalogerias Amin Karbasi MLT 73 2 0 14 Feb 2022
Generalization Bounds via Convex Analysis Gábor Lugosi Gergely Neu 77 29 0 10 Feb 2022
From Generalisation Error to Transportation-cost Inequalities and Back A. Esposito Michael C. Gastpar 74 7 0 08 Feb 2022
Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning L. P. Barnes Alex Dytso H. V. Poor FedML 57 19 0 04 Feb 2022
On generalization bounds for deep networks based on loss surface implicit regularization Masaaki Imaizumi Johannes Schmidt-Hieber ODL 68 3 0 12 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 84 8 0 09 Jan 2022
Generalization Bounds for Stochastic Gradient Langevin Dynamics: A Unified View via Information Leakage Analysis Bingzhe Wu Zhicong Liang Yatao Bian Chaochao Chen Junzhou Huang Yuan Yao 49 1 0 14 Dec 2021
Towards a Unified Information-Theoretic Framework for Generalization Mahdi Haghifam Gintare Karolina Dziugaite Shay Moran Daniel M. Roy 150 34 0 09 Nov 2021
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm Yuheng Bu Gholamali Aminian Laura Toni Miguel R. D. Rodrigues G. Wornell 64 14 0 02 Nov 2021
Optimizing Information-theoretical Generalization Bounds via Anisotropic Noise in SGLD Bohan Wang Huishuai Zhang Jieyu Zhang Qi Meng Wei Chen Tie-Yan Liu 24 1 0 26 Oct 2021
User-friendly introduction to PAC-Bayes bounds Pierre Alquier FedML 193 206 0 21 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 124 26 0 07 Oct 2021
Information-theoretic generalization bounds for black-box learning algorithms Hrayr Harutyunyan Maxim Raginsky Greg Ver Steeg Aram Galstyan 136 44 0 04 Oct 2021
Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning Haiyun He Hanshu Yan Vincent Y. F. Tan 97 11 0 03 Oct 2021
Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis Qi Chen Changjian Shui M. Marchand 99 44 0 29 Sep 2021
Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers Liam Hodgkinson Umut Simsekli Rajiv Khanna Michael W. Mahoney 77 23 0 02 Aug 2021
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information Gholamali Aminian Yuheng Bu Laura Toni M. Rodrigues G. Wornell 73 4 0 28 Jul 2021
Information-Theoretic Analysis of Epistemic Uncertainty in Bayesian Meta-learning Sharu Theresa Jose Sangwook Park Osvaldo Simeone PER UD UQCV 80 17 0 01 Jun 2021
Information Complexity and Generalization Bounds P. Banerjee Guido Montúfar 86 14 0 04 May 2021
Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels Hao Wang Rui Gao Flavio du Pin Calmon 81 18 0 05 Feb 2021
Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithms Gholamali Aminian Laura Toni M. Rodrigues 139 16 0 03 Feb 2021
Tighter expected generalization error bounds via Wasserstein distance Borja Rodríguez Gálvez Germán Bassi Ragnar Thobaben Mikael Skoglund 79 46 0 22 Jan 2021
An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning Sharu Theresa Jose Osvaldo Simeone 84 11 0 21 Jan 2021
Continuity of Generalized Entropy and Statistical Learning Aolin Xu 162 6 0 31 Dec 2020
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 130 51 0 20 Dec 2020
Individually Conditional Individual Mutual Information Bound on Generalization Error Ruida Zhou C. Tian Tie Liu 78 32 0 17 Dec 2020
Characterization of Excess Risk for Locally Strongly Convex Population Risk Mingyang Yi Ruoyu Wang Zhi-Ming Ma 44 2 0 04 Dec 2020
Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimization Sharu Theresa Jose Osvaldo Simeone G. Durisi 114 17 0 04 Nov 2020
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms Gholamali Aminian Laura Toni M. Rodrigues 83 31 0 23 Oct 2020
Fast-Rate Loss Bounds via Conditional Information Measures with Applications to Neural Networks Fredrik Hellström G. Durisi 95 2 0 22 Oct 2020