Prevalence of Neural Collapse during the terminal phase of deep learning training

18 August 2020

Papers citing "Prevalence of Neural Collapse during the terminal phase of deep learning training"

50 / 65 papers shown

Title
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 29 0 0 12 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 30 0 0 02 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang J. Hopcroft CLL 56 0 0 25 Apr 2025
You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data Shanshan Yan Zexi Li Chao-Xiang Wu Meng Pang Yang Lu Yan Yan Hanzi Wang FedML 55 0 0 10 Mar 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 55 7 0 20 Feb 2025
A Relative Homology Theory of Representation in Neural Networks Kosio Beshkov 89 0 0 17 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 48 3 0 31 Jan 2025
Technical report on label-informed logit redistribution for better domain generalization in low-shot classification with foundation models Behraj Khan T. Syed 52 1 0 29 Jan 2025
Fresh-CL: Feature Realignment through Experts on Hypersphere in Continual Learning Zhongyi Zhou Yaxin Peng Pin Yi Minjie Zhu Chaomin Shen 38 0 0 04 Jan 2025
Measuring Error Alignment for Decision-Making Systems Binxia Xu Antonis Bikakis Daniel Onah A. Vlachidis Luke Dickens 34 0 0 03 Jan 2025
CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning Junghun Oh Sungyong Baik Kyoung Mu Lee CLL 16 3 0 08 Oct 2024
Control-oriented Clustering of Visual Latent Representation Han Qi Haocheng Yin Heng Yang SSL 50 2 0 07 Oct 2024
Collapsed Language Models Promote Fairness Jingxuan Xu Wuyang Chen Linyi Li Yao Zhao Yunchao Wei 39 0 0 06 Oct 2024
SGW-based Multi-Task Learning in Vision Tasks Ruiyuan Zhang Yuyao Chen Yuchi Huo Jiaxiang Liu Dianbing Xi Jie Liu Chao Wu 20 0 0 03 Oct 2024
Formation of Representations in Neural Networks Liu Ziyin Isaac Chuang Tomer Galanti T. Poggio 34 4 0 03 Oct 2024
Linear Projections of Teacher Embeddings for Few-Class Distillation Noel Loo Fotis Iliopoulos Wei Hu Erik Vee 15 0 0 30 Sep 2024
Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment Naoya Hasegawa Issei Sato 31 0 0 26 Sep 2024
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training Kun Song Zhiquan Tan Bochao Zou Jiansheng Chen Huimin Ma Weiran Huang 30 0 0 25 Sep 2024
When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study Tomoyuki Obuchi Toshiyuki Tanaka 34 0 0 09 Sep 2024
Prototypical Partial Optimal Transport for Universal Domain Adaptation Yucheng Yang Xiang Gu Jian-jun Sun OT 27 11 0 02 Aug 2024
ProSub: Probabilistic Open-Set Semi-Supervised Learning with Subspace-Based Out-of-Distribution Detection Erik Wallin Lennart Svensson Fredrik Kahl Lars Hammarstrand OODD 26 1 0 16 Jul 2024
Geometric Analysis of Unconstrained Feature Models with $d=K$ Yi Shen Shao Gu 21 0 0 15 Jul 2024
Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detection F. Barbato Umberto Michieli J. Moon Pietro Zanuttigh Mete Ozay 30 2 0 01 Jul 2024
A deep neural network framework for dynamic multi-valued mapping estimation and its applications Geng Li Di Qiu L. Lui 19 0 0 29 Jun 2024
FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning Seongyoon Kim Minchan Jeong Sungnyun Kim Sungwoo Cho Sumyeong Ahn Se-Young Yun FedML 28 0 0 04 Jun 2024
Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection Yingwen Wu Ruiji Yu Xinwen Cheng Zhengbao He Xiaolin Huang OODD 60 1 0 28 May 2024
Spectral regularization for adversarially-robust representation learning Sheng Yang Jacob A. Zavatone-Veth C. Pehlevan AAML OOD 27 0 0 27 May 2024
WeiPer: OOD Detection using Weight Perturbations of Class Projections Maximilian Granz Manuel Heurich Tim Landgraf OODD 30 1 0 27 May 2024
Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets Arthur Jacot Alexandre Kaiser 23 0 0 27 May 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 23 0 0 29 Feb 2024
Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning Yuxiao Wen Arthur Jacot 37 6 0 12 Feb 2024
Pushing Boundaries: Mixup's Influence on Neural Collapse Quinn Fisher Haoming Meng V. Papyan AAML UQCV 22 5 0 09 Feb 2024
Manipulating Sparse Double Descent Ya Shi Zhang 9 0 0 19 Jan 2024
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 16 20 0 09 Oct 2023
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 26 1 0 13 Sep 2023
Novel Class Discovery for Long-tailed Recognition Zhang Chuyu Rui Xu Xuming He 16 15 0 06 Aug 2023
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants Yibo Yang Haobo Yuan Xiangtai Li Jianlong Wu Lefei Zhang Zhouchen Lin Philip H. S. Torr Dacheng Tao Bernard Ghanem CLL 8 8 0 03 Aug 2023
Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation Haoqi Wang Zhizhong Li Wayne Zhang 10 1 0 03 Aug 2023
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks Vignesh Kothapalli Tom Tirer Joan Bruna 19 10 0 04 Jul 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 23 4 0 06 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 6 6 0 06 Jun 2023
Reward Collapse in Aligning Large Language Models Ziang Song Tianle Cai Jason D. Lee Weijie J. Su ALM 15 22 0 28 May 2023
The emergence of clusters in self-attention dynamics Borjan Geshkovski Cyril Letrouit Yury Polyanskiy Philippe Rigollet 14 46 0 09 May 2023
Neural Collapse Inspired Federated Learning with Non-iid Data Chenxi Huang Liang Xie Yibo Yang Wenxiao Wang Binbin Lin D. Cai FedML 11 3 0 27 Mar 2023
Cut your Losses with Squentropy Like Hui M. Belkin S. Wright UQCV 13 7 0 08 Feb 2023
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning Yibo Yang Haobo Yuan Xiangtai Li Zhouchen Lin Philip H. S. Torr Dacheng Tao CLL 10 95 0 06 Feb 2023
Understanding Imbalanced Semantic Segmentation Through Neural Collapse Zhisheng Zhong Jiequan Cui Yibo Yang Xiaoyang Wu Xiaojuan Qi X. Zhang Jiaya Jia 119 44 0 03 Jan 2023
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 13 18 0 23 Dec 2022
A Law of Data Separation in Deep Learning Hangfeng He Weijie J. Su OOD 13 36 0 31 Oct 2022
Class Based Thresholding in Early Exit Semantic Segmentation Networks Alperen Görmez Erdem Koyuncu 12 5 0 27 Oct 2022