Prevalence of Neural Collapse during the terminal phase of deep learning training

18 August 2020

Papers citing "Prevalence of Neural Collapse during the terminal phase of deep learning training"

15 / 65 papers shown

Title
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 10 16 0 18 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 24 14 0 10 Oct 2022
Class-wise and reduced calibration methods Michael Panchenko Anes Benmerzoug Miguel de Benito Delgado 8 0 0 07 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 17 55 0 04 Oct 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 12 23 0 24 Jun 2022
DECK: Model Hardening for Defending Pervasive Backdoors Guanhong Tao Yingqi Liu Shuyang Cheng Shengwei An Zhuo Zhang Qiuling Xu Guangyu Shen Xiangyu Zhang AAML 10 7 0 18 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 10 69 0 08 Jun 2022
Do More Negative Samples Necessarily Hurt in Contrastive Learning? Pranjal Awasthi Nishanth Dikkala Pritish Kamath 17 40 0 03 May 2022
Federated Optimization of Smooth Loss Functions Ali Jadbabaie A. Makur Devavrat Shah FedML 8 7 0 06 Jan 2022
Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping Xuran Meng Jianfeng Yao 17 7 0 26 Nov 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 15 7 0 11 Oct 2021
The mathematics of adversarial attacks in AI -- Why deep learning is unstable despite the existence of stable neural networks Alexander Bastounis A. Hansen Verner Vlacic AAML OOD 16 26 0 13 Sep 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 9 93 0 02 Mar 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 112 162 0 29 Jan 2021
Convolutional Neural Networks Analyzed via Convolutional Sparse Coding V. Papyan Yaniv Romano Michael Elad 48 283 0 27 Jul 2016