Understanding Dimensional Collapse in Contrastive Self-supervised Learning

18 October 2021

Pascal Vincent

Papers citing "Understanding Dimensional Collapse in Contrastive Self-supervised Learning"

50 / 231 papers shown

Title
FNBench: Benchmarking Robust Federated Learning against Noisy Labels Xuefeng Jiang Jia Li Nannan Wu Z. F. Wu Xujing Li Sheng Sun Gang Xu Y. Wang Qi Li Min Liu FedML 47 3 0 10 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 113 0 0 02 May 2025
Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection Ziqing Fan Siyuan Du Shengchao Hu Pingjie Wang Li Shen Y. Zhang Dacheng Tao Y. Wang 41 1 0 29 Apr 2025
Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment Jinwoo Choi Seung-Woo Seo OffRL 29 0 0 21 Apr 2025
An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research Patrik Reizinger Randall Balestriero David Klindt Wieland Brendel 40 0 0 17 Apr 2025
Meta-Representational Predictive Coding: Biomimetic Self-Supervised Learning Alexander Ororbia Karl J. Friston Rajesh P. N. Rao SSL 52 0 0 22 Mar 2025
Divide and Conquer Self-Supervised Learning for High-Content Imaging Lucas Farndale Paul Henderson Edward W Roberts Ke Yuan SSL 58 0 0 10 Mar 2025
You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data Shanshan Yan Zexi Li Chao-Xiang Wu Meng Pang Yang Lu Yan Yan Hanzi Wang FedML 61 0 0 10 Mar 2025
Projection Head is Secretly an Information Bottleneck Zhuo Ouyang Kaiwen Hu Qi Zhang Yifei Wang Yisen Wang 46 0 0 01 Mar 2025
Estimating Control Barriers from Offline Data Hongzhan Yu Seth Farrell Ryo Yoshimitsu Zhizhen Qin Henrik I. Christensen Sicun Gao OffRL 52 3 0 21 Feb 2025
DivIL: Unveiling and Addressing Over-Invariance for Out-of- Distribution Generalization Jiaqi Wang Yuhang Zhou Zhixiong Zhang Qiguang Chen Yongqiang Chen James Cheng OODD 58 1 0 18 Feb 2025
Homeomorphism Prior for False Positive and Negative Problem in Medical Image Dense Contrastive Representation Learning Yuting He Boyu Wang Rongjun Ge Yang Chen Guanyu Yang Shuo Li MedIm AAML 67 2 0 07 Feb 2025
Video Representation Learning with Joint-Embedding Predictive Architectures Katrina Drozdov Ravid Shwartz-Ziv Yann LeCun AI4TS 74 1 0 14 Dec 2024
Variational Graph Contrastive Learning Shifeng Xie Jhony H. Giraldo 37 0 0 11 Nov 2024
Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning Md Rifat Arefin G. Subbaraj Nicolas Angelard-Gontier Yann LeCun Irina Rish Ravid Shwartz-Ziv C. Pal LRM 126 0 0 04 Nov 2024
Negative-Free Self-Supervised Gaussian Embedding of Graphs Yunhui Liu Tieke He Tao Zheng Jianhua Zhao SSL 38 1 0 02 Nov 2024
Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization Junlin He Jinxiao Du Wei Ma SSL 35 0 0 01 Nov 2024
Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization Junlin He Jinxiao Du Susu Xu Wei Ma 21 0 0 01 Nov 2024
HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms Kiran Kokilepersaud Seulgi Kim M. Prabhushankar Ghassan AlRegib 33 0 0 30 Oct 2024
Cross-Entropy Is All You Need To Invert the Data Generating Process Patrik Reizinger Alice Bizeul Attila Juhos Julia E. Vogt Randall Balestriero Wieland Brendel David Klindt SSL OOD BDL DRL 102 3 0 29 Oct 2024
Rethinking Positive Pairs in Contrastive Learning Jiantao Wu Shentong Mo Zhenhua Feng Sara Atito Josef Kitler Muhammad Awais SSL VLM 46 3 0 23 Oct 2024
Generalized Probabilistic Attention Mechanism in Transformers DongNyeong Heo Heeyoul Choi 54 0 0 21 Oct 2024
LAC: Graph Contrastive Learning with Learnable Augmentation in Continuous Space Zhenyu Lin Hongzheng Li Yingxia Shao Guanhua Ye Yawen Li Quanqing Xu 31 0 0 20 Oct 2024
An Efficient Contrastive Unimodal Pretraining Method for EHR Time Series Data Ryan N. King Shivesh Kodali Conrad Krueger Tianbao Yang Bobak J. Mortazavi AI4TS 23 0 0 11 Oct 2024
SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning Taha Bouhsine Imad El Aaroussi Atik Faysal Wang Huaxia 31 1 0 07 Oct 2024
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning Jianxiong Li Zhihao Wang Jinliang Zheng Xiaoai Zhou Guanming Wang ... Yu Liu Jingjing Liu Ya-Qin Zhang Junzhi Yu Xianyuan Zhan 38 2 0 02 Oct 2024
Decorrelation-based Self-Supervised Visual Representation Learning for Writer Identification Arkadip Maitra Shree Mitra Siladittya Manna Saumik Bhattacharya Umapada Pal SSL 25 0 0 02 Oct 2024
Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning Grzegorz Rype'sć Sebastian Cygert Tomasz Trzciñski Bartłomiej Twardowski 30 4 0 26 Sep 2024
Context-Aware Predictive Coding: A Representation Learning Framework for WiFi Sensing B. Barahimi Hina Tabassum M. Omer Omer Waqar SSL 27 0 0 16 Sep 2024
Exploiting Supervised Poison Vulnerability to Strengthen Self-Supervised Defense Jeremy A. Styborski Mingzhi Lyu Y. Huang Adams Kong 39 0 0 13 Sep 2024
Label-free Monitoring of Self-Supervised Learning Progress Isaac Xu Scott Lowe Thomas Trappenberg 32 1 0 10 Sep 2024
An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving S. Mohammed Alp Argun Nicolas Bonnotte Gerd Ascheid OffRL 28 0 0 02 Sep 2024
EEG-Language Modeling for Pathology Detection Sam Gijsen Kerstin Ritter 41 0 0 02 Sep 2024
Improving Nonlinear Projection Heads using Pretrained Autoencoder Embeddings Andreas Schliebitz Heiko Tapken Martin Atzmueller 35 0 0 25 Aug 2024
Detached and Interactive Multimodal Learning Yunfeng Fan Wenchao Xu Haozhao Wang Junhong Liu Song Guo 41 3 0 28 Jul 2024
On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition Zihu Wang Lingqiao Liu Scott Ricardo Figueroa Weston Samuel Tian Peng Li 29 2 0 19 Jul 2024
CLOVER: Context-aware Long-term Object Viewpoint- and Environment- Invariant Representation Learning Dongmyeong Lee Amanda Adkins Joydeep Biswas 40 0 0 12 Jul 2024
ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalities Julie Mordacq Léo Milecki Maria Vakalopoulou Steve Oudot Vicky Kalogeiton OffRL MedIm 37 3 0 04 Jul 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 33 5 0 28 Jun 2024
The Hidden Pitfalls of the Cosine Similarity Loss Andrew Draganov Sharvaree P. Vadgama Erik J. Bekkers SSL 31 0 0 24 Jun 2024
Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations? Mark Ibrahim David Klindt Randall Balestriero SSL 56 2 1 15 Jun 2024
Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit Lineghuan Meng Chuang Wang 23 1 0 11 Jun 2024
Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Loss Kiran Kokilepersaud Yavuz Yarici M. Prabhushankar Ghassan AlRegib 37 1 0 10 Jun 2024
iQRL -- Implicitly Quantized Representations for Sample-efficient Reinforcement Learning Aidan Scannell Kalle Kujanpää Yi Zhao Mohammadreza Nakhaei Arno Solin J. Pajarinen SSL 45 5 0 04 Jun 2024
DDA: Dimensionality Driven Augmentation Search for Contrastive Learning in Laparoscopic Surgery Yuning Zhou H. Badgery Matthew Read James Bailey Catherine E. Davey 45 1 0 03 Jun 2024
An Unsupervised Approach for Periodic Source Detection in Time Series B. U. Demirel Christian Holz AI4TS 42 1 0 01 Jun 2024
Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping Ziqing Fan Jiangchao Yao Ruipeng Zhang Lingjuan Lyu Ya-Qin Zhang Yanfeng Wang FedML 32 2 0 29 May 2024
Topological Perspectives on Optimal Multimodal Embedding Spaces Abdul Aziz Abdul Rahim BDL 39 0 0 29 May 2024
In-Context Symmetries: Self-Supervised Learning through Contextual World Models Sharut Gupta Chenyu Wang Yifei Wang Tommi Jaakkola Stefanie Jegelka 32 1 0 28 May 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 32 0 0 28 May 2024