Dimensionality compression and expansion in Deep Neural Networks

2 June 2019

Papers citing "Dimensionality compression and expansion in Deep Neural Networks"

36 / 36 papers shown

Title
Intermediate Layer Classifiers for OOD generalization Arnas Uselis Seong Joon Oh OOD 53 0 0 07 Apr 2025
Understanding How Nonlinear Layers Create Linearly Separable Features for Low-Dimensional Data Alec S. Xu Can Yaras Peng Wang Q. Qu 30 0 0 04 Jan 2025
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 84 0 0 02 Oct 2024
DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations Guogang Zhu Xuefeng Liu Jianwei Niu Shaojie Tang Xinghao Wu Jiayuan Zhang AI4CE 47 1 0 25 Jul 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi-An Ma 29 6 0 04 Jun 2024
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks Xue Geng Zhe Wang Chunyun Chen Qing Xu Kaixin Xu ... Zhenghua Chen M. Aly Jie Lin Min-man Wu Xiaoli Li 33 1 0 09 May 2024
Neural Network Approximation for Pessimistic Offline Reinforcement Learning Di Wu Yuling Jiao Li Shen Haizhao Yang Xiliang Lu OffRL 29 1 0 19 Dec 2023
On original and latent space connectivity in deep neural networks Boyang Gu Anastasia Borovykh GNN 3DPC 27 1 0 12 Nov 2023
Outlier Dimensions Encode Task-Specific Knowledge William Rudman Catherine Chen Carsten Eickhoff 11 4 0 26 Oct 2023
Bridging Information-Theoretic and Geometric Compression in Language Models Emily Cheng Corentin Kervadec Marco Baroni 34 16 0 20 Oct 2023
A simple connection from loss flatness to compressed neural representations Shirui Chen Stefano Recanatesi E. Shea-Brown 15 0 0 03 Oct 2023
Exploring Learned Representations of Neural Networks with Principal Component Analysis Amit Harlev A. Engel P. Stinis Tony Chiang FAtt 17 0 0 27 Sep 2023
How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model Francesco Cagnetta Leonardo Petrini Umberto M. Tomasini Alessandro Favero M. Wyart BDL 30 22 0 05 Jul 2023
Stable Anisotropic Regularization William Rudman Carsten Eickhoff 25 6 0 30 May 2023
A Rainbow in Deep Network Black Boxes Florentin Guth Brice Ménard G. Rochette S. Mallat 22 10 0 29 May 2023
Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training Xinwei Ou Zhangxin Chen Ce Zhu Yipeng Liu 21 4 0 22 Mar 2023
Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation Simone Ciceri Lorenzo Cassani Matteo Osella P. Rotondo P. Pizzochero M. Gherardi 31 7 0 09 Mar 2023
Relating Regularization and Generalization through the Intrinsic Dimension of Activations Bradley Brown Jordan Juravsky Anthony L. Caterini G. Loaiza-Ganem 31 3 0 23 Nov 2022
How deep convolutional neural networks lose spatial information with training Umberto M. Tomasini Leonardo Petrini Francesco Cagnetta M. Wyart 41 9 0 04 Oct 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 22 41 0 19 Sep 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 39 23 0 24 Jun 2022
Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods Yuchen Lu Zhen Liu A. Baratin Romain Laroche Aaron C. Courville Alessandro Sordoni SSL 23 0 0 02 Jun 2022
CNNs Avoid Curse of Dimensionality by Learning on Patches Vamshi C. Madala S. Chandrasekaran Jason Bunk UQCV 27 5 0 22 May 2022
Origami in N dimensions: How feed-forward networks manufacture linear separability Christian Keup M. Helias 13 8 0 21 Mar 2022
Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphs David Bonet Antonio Ortega Javier Ruiz-Hidalgo Sarath Shekkizhar GNN 33 7 0 18 Oct 2021
Channel-Wise Early Stopping without a Validation Set via NNK Polytope Interpolation David Bonet Antonio Ortega Javier Ruiz-Hidalgo Sarath Shekkizhar 35 15 0 27 Jul 2021
The Foes of Neural Network's Data Efficiency Among Unnecessary Input Dimensions Vanessa D’Amario S. Srivastava Tomotake Sasaki Xavier Boix AAML 21 2 0 13 Jul 2021
Relative stability toward diffeomorphisms indicates performance in deep nets Leonardo Petrini Alessandro Favero Mario Geiger M. Wyart OOD 36 15 0 06 May 2021
A Theoretical-Empirical Approach to Estimating Sample Complexity of DNNs Devansh Bisla Apoorva Nandini Saridena A. Choromańska 28 8 0 05 May 2021
Deep Nonparametric Regression on Approximate Manifolds: Non-Asymptotic Error Bounds with Polynomial Prefactors Yuling Jiao Guohao Shen Yuanyuan Lin Jian Huang 36 50 0 14 Apr 2021
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini M. Wyart DRL 23 11 0 30 Dec 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 52 34 0 22 Jul 2020
Weakly-correlated synapses promote dimension reduction in deep neural networks Jianwen Zhou Haiping Huang 11 6 0 20 Jun 2020
Emergence of Separable Manifolds in Deep Language Representations Jonathan Mamou Hang Le Miguel Angel del Rio Cory Stephenson Hanlin Tang Yoon Kim SueYeon Chung AAML AI4CE 14 38 0 01 Jun 2020
Internal representation dynamics and geometry in recurrent neural networks Stefan Horoi Guillaume Lajoie Guy Wolf GNN 10 5 0 09 Jan 2020
Intrinsic dimension estimation for locally undersampled data Vittorio Erba M. Gherardi P. Rotondo 14 30 0 18 Jun 2019