Entropy and mutual information in models of deep neural networks

Entropy and mutual information in models of deep neural networks

24 May 2018

Marylou Gabrié

Clément Luneau

Florent Krzakala

Lenka Zdeborová

Papers citing "Entropy and mutual information in models of deep neural networks"

15 / 15 papers shown

Title
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime Francesco Camilli D. Tieplova Eleonora Bergamin Jean Barbier 97 0 0 06 May 2025
ACMamba: Fast Unsupervised Anomaly Detection via An Asymmetrical Consensus State Space Model Guanchun Wang Xiangrong Zhang Yifei Zhang Zelin Peng Tianyang Zhang Xu Tang Licheng Jiao Mamba 37 0 0 16 Apr 2025
High-entropy Advantage in Neural Networks' Generalizability Entao Yang X. Zhang Yue Shang Ge Zhang AI4CE 58 0 0 17 Mar 2025
Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics Yehonatan Avidan Qianyi Li H. Sompolinsky 60 8 0 08 Sep 2023
GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction Yucheng Shi Yushun Dong Qiaoyu Tan Jundong Li Ninghao Liu 40 24 0 18 Aug 2023
Neural-prior stochastic block model O. Duranthon L. Zdeborová 30 3 0 17 Mar 2023
Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group J. Erdmenger Kevin T. Grosvenor R. Jefferson 46 17 0 14 Jul 2021
Optimizing Information Loss Towards Robust Neural Networks Philip Sperl Konstantin Böttinger AAML 13 3 0 07 Aug 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 33 100 0 25 Jun 2020
On the Maximum Mutual Information Capacity of Neural Architectures Brandon Foggo Nan Yu TPM 15 3 0 10 Jun 2020
On Information Plane Analyses of Neural Network Classifiers -- A Review Bernhard C. Geiger 24 50 0 21 Mar 2020
Inference in Multi-Layer Networks with Matrix-Valued Unknowns Parthe Pandit Mojtaba Sahraee-Ardakan S. Rangan P. Schniter A. Fletcher 18 6 0 26 Jan 2020
Modelling the influence of data structure on learning in neural networks: the hidden manifold model Sebastian Goldt M. Mézard Florent Krzakala Lenka Zdeborová BDL 21 51 0 25 Sep 2019
Convergence of Smoothed Empirical Measures with Applications to Entropy Estimation Ziv Goldfeld Kristjan Greenewald Yury Polyanskiy Jonathan Niles-Weed 21 64 0 30 May 2019
Asymptotics of MAP Inference in Deep Networks Parthe Pandit Mojtaba Sahraee S. Rangan A. Fletcher 8 21 0 01 Mar 2019