Neural collapse with unconstrained features

23 November 2020

Papers citing "Neural collapse with unconstrained features"

50 / 92 papers shown

Title
Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model G. Andriopoulos Soyuj Jung Basnet Juan Guevara Li Guo Keith Ross 22 0 0 14 May 2025
On the Geometry of Semantics in Next-token Prediction Yize Zhao Christos Thrampoulidis 23 0 0 13 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang J. Hopcroft CLL 56 0 0 25 Apr 2025
DualCP: Rehearsal-Free Domain-Incremental Learning via Dual-Level Concept Prototype Qiang Wang Yuhang He Songlin Dong Xiang Song Jizhou Han Haoyu Luo Yihong Gong CLL 62 0 0 23 Mar 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 38 0 0 28 Feb 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 55 8 0 20 Feb 2025
Dependence Induced Representations Xiangxiang Xu Lizhong Zheng 57 1 0 22 Nov 2024
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame Evan Markou Thalaiyasingam Ajanthan Stephen Gould 26 0 0 02 Nov 2024
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective Haixiang Sun Ye Shi 31 0 0 30 Oct 2024
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 34 2 0 30 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 91 14 0 11 Oct 2024
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 31 1 0 07 Oct 2024
The Prevalence of Neural Collapse in Neural Multivariate Regression G. Andriopoulos Zixuan Dong Li Guo Zifan Zhao Keith Ross 47 3 0 06 Sep 2024
Beyond Unconstrained Features: Neural Collapse for Shallow Neural Networks with General Data Wanli Hong Shuyang Ling 29 3 0 03 Sep 2024
Geometric Analysis of Unconstrained Feature Models with $d=K$ Yi Shen Shao Gu 27 0 0 15 Jul 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi-An Ma 29 6 0 04 Jun 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 46 12 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 32 7 0 23 May 2024
Stationary Representations: Optimally Approximating Compatibility and Implications for Improved Model Replacements Niccoló Biondi F. Pernici Simone Ricci A. Bimbo 22 2 0 04 May 2024
Progressive Feedforward Collapse of ResNet Training Sicong Wang Kuo Gai Shihua Zhang AI4CE 33 4 0 02 May 2024
Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model Connall Garrod Jonathan P. Keating 33 8 0 09 Apr 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 31 0 0 29 Feb 2024
Average gradient outer product as a mechanism for deep neural collapse Daniel Beaglehole Peter Súkeník Marco Mondelli Misha Belkin AI4CE 27 12 0 21 Feb 2024
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse Bradley T. Baker Ba Pearlmutter Robyn L. Miller Vince D. Calhoun Sergey Plis AI4CE 11 2 0 09 Feb 2024
Pushing Boundaries: Mixup's Influence on Neural Collapse Quinn Fisher Haoming Meng V. Papyan AAML UQCV 36 5 0 09 Feb 2024
Cross Entropy versus Label Smoothing: A Neural Collapse Perspective Li Guo Keith Ross Zifan Zhao G. Andriopoulos Shuyang Ling Yufeng Xu Zixuan Dong UQCV NoLa 25 9 0 06 Feb 2024
Enhancing Compositional Generalization via Compositional Feature Alignment Haoxiang Wang Haozhe Si Huajie Shao Han Zhao 40 1 0 05 Feb 2024
FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data Zikai Xiao Zihan Chen Liyinglan Liu Yang Feng Jian Wu Wanlu Liu Joey Tianyi Zhou Howard H. Yang Zuo-Qiang Liu FedML 37 6 0 17 Jan 2024
On the Robustness of Neural Collapse and the Neural Collapse of Robustness Jingtong Su Ya Shi Zhang Nikolaos Tsilivis Julia Kempe AAML 29 4 0 13 Nov 2023
Detecting Out-of-Distribution Through the Lens of Neural Collapse Litian Liu Yao Qin OODD 36 5 0 02 Nov 2023
Combating Representation Learning Disparity with Geometric Harmonization Zhihan Zhou Jiangchao Yao Feng Hong Ya-Qin Zhang Bo Han Yanfeng Wang 25 12 0 26 Oct 2023
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 22 8 0 24 Oct 2023
Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges Peifeng Gao Qianqian Xu Yibo Yang Peisong Wen Huiyang Shao Zhiyong Yang Bernard Ghanem Qingming Huang AAML 18 3 0 12 Oct 2023
NECO: NEural Collapse Based Out-of-distribution detection Mouin Ben Ammar Nacim Belkhir Sebastian Popescu Antoine Manzanera Gianni Franchi OODD 29 18 0 10 Oct 2023
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 32 23 0 09 Oct 2023
An Investigation of Representation and Allocation Harms in Contrastive Learning Subha Maity Mayank Agarwal Mikhail Yurochkin Yuekai Sun 25 2 0 02 Oct 2023
Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss Jaidev Gill V. Vakilian Christos Thrampoulidis 6 1 0 02 Oct 2023
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Y. Ma VLM MLLM CLL 77 78 0 19 Sep 2023
Neural Collapse for Unconstrained Feature Model under Cross-entropy Loss with Imbalanced Data Wanli Hong Shuyang Ling 21 14 0 18 Sep 2023
Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay Leyan Pan Xinyuan Cao 19 3 0 09 Sep 2023
Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networks Liam Parker Emre Onal Anton Stengel Jake Intrater 25 11 0 05 Aug 2023
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants Yibo Yang Haobo Yuan Xiangtai Li Jianlong Wu Lefei Zhang Zhouchen Lin Philip H. S. Torr Dacheng Tao Bernard Ghanem CLL 31 8 0 03 Aug 2023
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks Vignesh Kothapalli Tom Tirer Joan Bruna 34 10 0 04 Jul 2023
Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations Yongyi Yang Jacob Steinhardt Wei Hu 21 10 0 29 Jun 2023
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching Ganesh Ramachandra Kini V. Vakilian Tina Behnia Jaidev Gill Christos Thrampoulidis 17 1 0 13 Jun 2023
Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalization J. Haas William Yolland B. Rabus OODD 19 0 0 07 Jun 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 33 4 0 06 Jun 2023
Neural (Tangent Kernel) Collapse Mariia Seleznova Dana Weitzner Raja Giryes Gitta Kutyniok H. Chou 21 6 0 25 May 2023
Feature Collapse T. Laurent J. V. Brecht Xavier Bresson 17 3 0 25 May 2023
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model Peter Súkeník Marco Mondelli Christoph H. Lampert 26 21 0 22 May 2023