Understanding Dimensional Collapse in Contrastive Self-supervised Learning

18 October 2021

Pascal Vincent

Papers citing "Understanding Dimensional Collapse in Contrastive Self-supervised Learning"

50 / 231 papers shown

Title
Relational Self-supervised Distillation with Compact Descriptors for Image Copy Detection Juntae Kim Sungwon Woo Jongho Nang 42 1 0 28 May 2024
Unveiling Key Aspects of Fine-Tuning in Sentence Embeddings: A Representation Rank Analysis Euna Jung Jaeill Kim Jungmin Ko Jinwoo Park Wonjong Rhee 50 0 0 18 May 2024
JOSENet: A Joint Stream Embedding Network for Violence Detection in Surveillance Videos Pietro Nardelli Danilo Comminiello 19 0 0 05 May 2024
Modeling Caption Diversity in Contrastive Vision-Language Pretraining Samuel Lavoie Polina Kirichenko Mark Ibrahim Mahmoud Assran Andrew Gordon Wilson Aaron Courville Nicolas Ballas CLIP VLM 64 19 0 30 Apr 2024
Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G Walid Saad Omar Hashash Christo Kurisummoottil Thomas Christina Chaccour Merouane Debbah N. Mandayam Zhu Han AI4CE 39 23 0 29 Apr 2024
Retrieval-Oriented Knowledge for Click-Through Rate Prediction Huanshuo Liu Bo Chen Menghui Zhu Jianghao Lin Jiarui Qin Yang Yang Hao Zhang Ruiming Tang 27 3 0 28 Apr 2024
An Experimental Comparison Of Multi-view Self-supervised Methods For Music Tagging Gabriel Meseguer-Brocal Dorian Desblancs Romain Hennequin SSL 41 3 0 14 Apr 2024
Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck Nathan Godey Eric Villemonte de la Clergerie Benoît Sagot 48 8 0 11 Apr 2024
HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphs Sunwoo Kim Shinhwan Kang Fanchen Bu Soo Yong Lee Jaemin Yoo Kijung Shin SSL 29 11 0 31 Mar 2024
Noise-Robust Keyword Spotting through Self-supervised Pretraining Jacob Mork H. S. Bovbjerg Gergely Kiss Zheng-Hua Tan 24 3 0 27 Mar 2024
Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data Xinting Liao Weiming Liu Chaochao Chen Pengyang Zhou Fengyuan Yu Huabin Zhu Binhui Yao Tao Wang Xiaolin Zheng Yanchao Tan FedML 43 7 0 25 Mar 2024
Contrastive Balancing Representation Learning for Heterogeneous Dose-Response Curves Estimation Minqin Zhu Anpeng Wu Haoxuan Li Ruoxuan Xiong Bo Li ... Xuan Qin Peng Zhen Jiecheng Guo Fei Wu Kun Kuang CML 39 9 0 21 Mar 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 41 10 0 18 Mar 2024
Pantypes: Diverse Representatives for Self-Explainable Models R. Kjærsgaard Ahcène Boubekki Line H. Clemmensen 28 2 0 14 Mar 2024
Joint-Embedding Masked Autoencoder for Self-supervised Learning of Dynamic Functional Connectivity from the Human Brain Jungwon Choi Hyungi Lee Byung-Hoon Kim Juho Lee 77 0 0 11 Mar 2024
Self-Supervised Multiple Instance Learning for Acute Myeloid Leukemia Classification Salome Kazeminia Max Joosten D. Bosnacki Carsten Marr 22 2 0 08 Mar 2024
Rethinking The Uniformity Metric in Self-Supervised Learning Xianghong Fang Jian Li Qiang Sun Benyou Wang SSL 12 0 0 01 Mar 2024
DeepSet SimCLR: Self-supervised deep sets for improved pathology representation learning David Torpey Richard Klein SSL 42 1 0 23 Feb 2024
Ads Recommendation in a Collapsed and Entangled World Junwei Pan Wei Xue Ximei Wang Haibin Yu Xun Liu Shijie Quan Xueming Qiu Dapeng Liu Lei Xiao Jie Jiang 45 5 0 22 Feb 2024
CCFC++: Enhancing Federated Clustering through Feature Decorrelation Jie Yan Jing Liu Yi-Zi Ning Zhonghan Zhang FedML 19 2 0 20 Feb 2024
EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs Navid Mohammadi Foumani G. Mackellar Soheila Ghane Saad Irtza Nam Nguyen Mahsa Salehi 14 14 0 17 Feb 2024
WERank: Towards Rank Degradation Prevention for Self-Supervised Learning Using Weight Regularization Ali Saheb Pasand Reza Moravej Mahdi Biparva Ali Ghodsi 34 2 0 14 Feb 2024
Graph Inference Acceleration by Learning MLPs on Graphs without Supervision Zehong Wang Zheyuan Zhang Chuxu Zhang Yanfang Ye 20 5 0 14 Feb 2024
Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening Hoyong Kim Semi Lee Kangil Kim CLL 24 2 0 26 Jan 2024
Memorization in Self-Supervised Learning Improves Downstream Generalization Wenhao Wang Muhammad Ahmad Kaleem Adam Dziedzic Michael Backes Nicolas Papernot Franziska Boenisch SSL 17 9 0 19 Jan 2024
LDReg: Local Dimensionality Regularized Self-Supervised Learning Hanxun Huang R. Campello S. Erfani Xingjun Ma Michael E. Houle James Bailey 30 5 0 19 Jan 2024
Bridging State and History Representations: Understanding Self-Predictive RL Tianwei Ni Benjamin Eysenbach Erfan Seyedsalehi Michel Ma Clement Gehring Aditya Mahajan Pierre-Luc Bacon AI4TS AI4CE 22 20 0 17 Jan 2024
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data Yuhui Zhang Elaine Sui Serena Yeung-Levy 29 9 0 16 Jan 2024
Generic Knowledge Boosted Pre-training For Remote Sensing Images Ziyue Huang Mingming Zhang Yuan Gong Qingjie Liu Yunhong Wang VLM 28 14 0 09 Jan 2024
Understanding Representation Learnability of Nonlinear Self-Supervised Learning Ruofeng Yang Xiangyuan Li Bo Jiang Shuai Li SSL MLT 45 2 0 06 Jan 2024
Analyzing Local Representations of Self-supervised Vision Transformers Ani Vanyan Alvard Barseghyan Hakob Tamazyan Vahan Huroyan Hrant Khachatrian Martin Danelljan 39 3 0 31 Dec 2023
3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression from Longitudinal OCTs T. Emre A. Chakravarty Antoine Rivail Dmitrii Lachinov Oliver Leingang ... S. Sivaprasad Daniel Rueckert A. Lotery U. Schmidt-Erfurth Hrvoje Bogunović MedIm 27 3 0 28 Dec 2023
Self-Supervised Learning for Few-Shot Bird Sound Classification Ilyass Moummad Romain Serizel Nicolas Farrugia SSL 13 9 0 25 Dec 2023
Layerwise complexity-matched learning yields an improved model of cortical area V2 Nikhil Parthasarathy Olivier J. Hénaff Eero P. Simoncelli 34 1 0 18 Dec 2023
Prototypical Self-Explainable Models Without Re-training Srishti Gautam Ahcène Boubekki Marina M.-C. Höhne Michael C. Kampffmeyer 26 2 0 13 Dec 2023
LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures Vimal Thilak Chen Huang Omid Saremi Laurent Dinh Hanlin Goh Preetum Nakkiran Josh Susskind Etai Littwin 18 7 0 07 Dec 2023
Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation Randall Balestriero Romain Cosentino Sarath Shekkizhar 28 2 0 04 Dec 2023
Revisiting Supervision for Continual Representation Learning Daniel Marczak Sebastian Cygert Tomasz Trzciñski Bartlomiej Twardowski SSL CLL 32 0 0 22 Nov 2023
Robust Contrastive Learning With Theory Guarantee Ngoc N. Tran Lam C. Tran Hoang Phan Anh-Vu Bui Tung Pham Toan M. Tran Dinh Q. Phung Trung Le SSL NoLa 23 0 0 16 Nov 2023
Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations Cian Eastwood Julius von Kügelgen Linus Ericsson Diane Bouchacourt Pascal Vincent Bernhard Schölkopf Mark Ibrahim 34 10 0 15 Nov 2023
APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation Mingjia Yin Hao Wang Xiang Xu Likang Wu Sirui Zhao Wei Guo Yong Liu Ruiming Tang Defu Lian Enhong Chen LMTD AI4TS HAI 35 25 0 06 Nov 2023
Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks Rouzbeh Meshkinnejad Jie Mei Daniel Lizotte Y. Mohsenzadeh SSL CLL 39 3 0 02 Nov 2023
Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Data Cheng-Hao Tu Hong-You Chen Zheda Mai Jike Zhong Vardaan Pahuja Tanya Berger-Wolf Song Gao Charles V. Stewart Yu-Chuan Su Wei-Lun Chao CLL 34 3 0 02 Nov 2023
Identifiable Contrastive Learning with Automatic Feature Importance Discovery Qi Zhang Yifei Wang Yisen Wang 26 12 0 29 Oct 2023
Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approach Ali Javidani Mohammad Amin Sadeghi Babak Nadjar Araabi 25 0 0 28 Oct 2023
Unveiling the Potential of Probabilistic Embeddings in Self-Supervised Learning Denis Janiak Jakub Binkowski Piotr Bielak Tomasz Kajdanowicz SSL UQCV 20 0 0 27 Oct 2023
Combining Behaviors with the Successor Features Keyboard Wilka Carvalho Andre Saraiva Angelos Filos Andrew Kyle Lampinen Loic Matthey Richard L. Lewis Honglak Lee Satinder Singh Danilo Jimenez Rezende Daniel Zoran 8 3 0 24 Oct 2023
Learning with Unmasked Tokens Drives Stronger Vision Learners Taekyung Kim Sanghyuk Chun Byeongho Heo Dongyoon Han SSL 36 1 0 20 Oct 2023
Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift Yihao Xue Siddharth Joshi Dang Nguyen Baharan Mirzasoleiman VLM 29 4 0 08 Oct 2023
On the Embedding Collapse when Scaling up Recommendation Models Xingzhuo Guo Kai Yan Ximei Wang Baixu Chen Haohan Wang Mingsheng Long 13 21 0 06 Oct 2023