The Low-Rank Simplicity Bias in Deep Networks

18 March 2021

Papers citing "The Low-Rank Simplicity Bias in Deep Networks"

29 / 29 papers shown

Title
Make Haste Slowly: A Theory of Emergent Structured Mixed Selectivity in Feature Learning ReLU Networks Devon Jarvis Richard Klein Benjamin Rosman Andrew M. Saxe MLT 64 1 0 08 Mar 2025
Sebra: Debiasing Through Self-Guided Bias Ranking Adarsh Kappiyath Abhra Chaudhuri Ajay Jaiswal Ziquan Liu Yunpeng Li Xiatian Zhu L. Yin CML 106 1 1 30 Jan 2025
Self-Assembly of a Biologically Plausible Learning Circuit Q. Liao Liu Ziyin Yulu Gan Brian Cheung Mark Harnett Tomaso Poggio 52 0 0 31 Dec 2024
SHAP values via sparse Fourier representation Ali Gorji Andisheh Amrollahi A. Krause FAtt 32 0 0 08 Oct 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Pretraining with Random Noise for Fast and Robust Learning without Weight Transport Jeonghwan Cheon Sang Wan Lee Se-Bum Paik OOD 147 1 0 27 May 2024
Learned feature representations are biased by complexity, learning order, position, and more Andrew Kyle Lampinen Stephanie C. Y. Chan Katherine Hermann AI4CE FaML SSL OOD 34 6 0 09 May 2024
Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition Xitong Zhang Ismail R. Alkhouri Rongrong Wang 35 0 0 06 May 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 94 18 0 04 Mar 2024
The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank Amitay Bar Rotem Mulayoff T. Michaeli Ronen Talmon 61 0 0 21 Feb 2024
Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval Xiu-Shen Wei Yang Shen Xuhao Sun Peng Wang Yuxin Peng 22 10 0 21 Nov 2023
Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks Woojin Cho Kookjin Lee Donsub Rim Noseong Park AI4CE PINN 26 16 0 14 Oct 2023
Robust low-rank training via approximate orthonormal constraints Dayana Savostianova Emanuele Zangrando Gianluca Ceruti Francesco Tudisco 24 9 0 02 Jun 2023
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models Suzanna Parkinson Greg Ongie Rebecca Willett 60 6 0 24 May 2023
Do deep neural networks have an inbuilt Occam's razor? Chris Mingard Henry Rees Guillermo Valle Pérez A. Louis UQCV BDL 19 15 0 13 Apr 2023
Delving Deep into Simplicity Bias for Long-Tailed Image Recognition Xiu-Shen Wei Xuhao Sun Yang Shen Anqi Xu Peng Wang Faen Zhang 36 1 0 07 Feb 2023
Evolution of Neural Tangent Kernels under Benign and Adversarial Training Noel Loo Ramin Hasani Alexander Amini Daniela Rus AAML 28 13 0 21 Oct 2022
Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation Thomas Duboudin Emmanuel Dellandréa Corentin Abgrall Gilles Hénaff Limin Chen TTA 21 1 0 17 Oct 2022
Overcoming the Spectral Bias of Neural Value Approximation Ge Yang Anurag Ajay Pulkit Agrawal 32 25 0 09 Jun 2022
Machine Learning and Deep Learning -- A review for Ecologists Maximilian Pichler F. Hartig 39 127 0 11 Apr 2022
Neural Fields in Visual Computing and Beyond Yiheng Xie Towaki Takikawa Shunsuke Saito Or Litany Shiqin Yan Numair Khan Federico Tombari James Tompkin Vincent Sitzmann Srinath Sridhar 3DH 49 613 0 22 Nov 2021
In Search of Probeable Generalization Measures Jonathan Jaegerman Khalil Damouni M. M. Ankaralı Konstantinos N. Plataniotis 19 2 0 23 Oct 2021
How Does Adversarial Fine-Tuning Benefit BERT? J. Ebrahimi Hao Yang Wei Zhang AAML 20 4 0 31 Aug 2021
Can contrastive learning avoid shortcut solutions? Joshua Robinson Li Sun Ke Yu Kayhan Batmanghelich Stefanie Jegelka S. Sra SSL 19 141 0 21 Jun 2021
Layer Folding: Neural Network Depth Reduction using Activation Linearization Amir Ben Dror Niv Zehngut Avraham Raviv E. Artyomov Ran Vitek R. Jevnisek 26 20 0 17 Jun 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 231 4,469 0 23 Jan 2020
Why bigger is not always better: on finite and infinite neural networks Laurence Aitchison 175 51 0 17 Oct 2019
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,216 0 16 Nov 2016
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 269 5,326 0 05 Nov 2016