A Law of Data Separation in Deep Learning

31 October 2022

Weijie J. Su

Papers citing "A Law of Data Separation in Deep Learning"

29 / 29 papers shown

Title
Understanding How Nonlinear Layers Create Linearly Separable Features for Low-Dimensional Data Alec S. Xu Can Yaras Peng Wang Q. Qu 23 0 0 04 Jan 2025
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 34 1 0 30 Oct 2024
Collapsed Language Models Promote Fairness Jingxuan Xu Wuyang Chen Linyi Li Yao Zhao Yunchao Wei 39 0 0 06 Oct 2024
The Prevalence of Neural Collapse in Neural Multivariate Regression G. Andriopoulos Zixuan Dong Li Guo Zifan Zhao Keith Ross 28 2 0 06 Sep 2024
A Law of Next-Token Prediction in Large Language Models Hangfeng He Weijie J. Su 24 5 0 24 Aug 2024
A spring-block theory of feature learning in deep neural networks Chengzhi Shi Liming Pan Ivan Dokmanić AI4CE 28 1 0 28 Jul 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip H. S. Torr Bernard Ghanem 30 2 0 07 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 27 12 0 06 Jun 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 33 11 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 30 5 0 23 May 2024
Neural Collapse Meets Differential Privacy: Curious Behaviors of NoisyGD with Near-perfect Representation Learning Chendi Wang Yuqing Zhu Weijie J. Su Yu-Xiang Wang AAML 35 4 0 14 May 2024
Progressive Feedforward Collapse of ResNet Training Sicong Wang Kuo Gai Shihua Zhang AI4CE 25 4 0 02 May 2024
Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model Connall Garrod Jonathan P. Keating 18 8 0 09 Apr 2024
Average gradient outer product as a mechanism for deep neural collapse Daniel Beaglehole Peter Súkeník Marco Mondelli Misha Belkin AI4CE 25 12 0 21 Feb 2024
On the Robustness of Neural Collapse and the Neural Collapse of Robustness Jingtong Su Ya Shi Zhang Nikolaos Tsilivis Julia Kempe AAML 15 4 0 13 Nov 2023
Detecting Out-of-Distribution Through the Lens of Neural Collapse Litian Liu Yao Qin OODD 23 5 0 02 Nov 2023
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 8 6 0 24 Oct 2023
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks Vignesh Kothapalli Tom Tirer Joan Bruna 22 10 0 04 Jul 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 23 4 0 06 Jun 2023
The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks Can Yaras P. Wang Wei Hu Zhihui Zhu Laura Balzano Qing Qu 20 17 0 01 Jun 2023
White-Box Transformers via Sparse Rate Reduction Yaodong Yu Sam Buchanan Druv Pai Tianzhe Chu Ziyang Wu Shengbang Tong B. Haeffele Y. Ma ViT 16 80 0 01 Jun 2023
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model Peter Súkeník Marco Mondelli Christoph H. Lampert 13 20 0 22 May 2023
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 16 18 0 23 Dec 2022
Perturbation Analysis of Neural Collapse Tom Tirer Haoxiang Huang Jonathan Niles-Weed AAML 27 23 0 29 Oct 2022
Neural Networks as Paths through the Space of Representations Richard D. Lange Devin Kwok Jordan K Matelsky Xinyue Wang David Rolnick Konrad Paul Kording 19 4 0 22 Jun 2022
Nearest Class-Center Simplification through Intermediate Layers Ido Ben-Shaul S. Dekel 32 26 0 21 Jan 2022
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 114 162 0 29 Jan 2021
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,214 0 17 Apr 2017
Densely Connected Convolutional Networks Gao Huang Zhuang Liu L. V. D. van der Maaten Kilian Q. Weinberger PINN 3DV 244 35,884 0 25 Aug 2016