Understanding Dimensional Collapse in Contrastive Self-supervised Learning

18 October 2021

Pascal Vincent

Papers citing "Understanding Dimensional Collapse in Contrastive Self-supervised Learning"

31 / 231 papers shown

Title
Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding Tianyang Hu Zhili Liu Fengwei Zhou Wenjia Wang Weiran Huang SSL 36 26 0 30 May 2022
Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods Randall Balestriero Yann LeCun SSL 18 129 0 23 May 2022
Toward a Geometrical Understanding of Self-supervised Contrastive Learning Romain Cosentino Anirvan M. Sengupta Salman Avestimehr Mahdi Soltanolkotabi Antonio Ortega Ted Willke Mariano Tepper SSL 37 17 0 13 May 2022
Posterior Collapse of a Linear Latent Variable Model Zihao W. Wang Liu Ziyin BDL 25 21 0 09 May 2022
Reducing Predictive Feature Suppression in Resource-Constrained Contrastive Image-Caption Retrieval Maurits J. R. Bleeker Andrew Yates Maarten de Rijke 41 4 0 28 Apr 2022
On the Representation Collapse of Sparse Mixture of Experts Zewen Chi Li Dong Shaohan Huang Damai Dai Shuming Ma ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei MoMe MoE 37 96 0 20 Apr 2022
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey Kento Nozawa Issei Sato AI4TS 19 4 0 18 Apr 2022
Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning Mayee F. Chen Daniel Y. Fu A. Narayan Michael Zhang Zhao-quan Song Kayvon Fatahalian Christopher Ré SSL 19 46 0 15 Apr 2022
Deep Normed Embeddings for Patient Representation Thesath Nanayakkara G. Clermont C. Langmead D. Swigon AI4TS 25 1 0 12 Apr 2022
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification Samuel Lavoie Christos Tsirigotis Max Schwarzer Ankit Vani Michael Noukhovitch Kenji Kawaguchi Aaron C. Courville SSL 24 17 0 01 Apr 2022
Contrasting the landscape of contrastive and non-contrastive learning Ashwini Pokle Jinjin Tian Yuchen Li Andrej Risteski SSL 30 28 0 29 Mar 2022
Probabilistic Spherical Discriminant Analysis: An Alternative to PLDA for length-normalized embeddings Niko Brummer Albert Swart Ladislav Movsner Anna Silnova Oldvrich Plchot Themos Stafylakis Lukávs Burget 18 5 0 28 Mar 2022
On Understanding and Mitigating the Dimensional Collapse of Graph Contrastive Learning: a Non-Maximum Removal Approach Jiawei Sun Ruoxin Chen Jie Li Chentao Wu Yue Ding Junchi Yan 29 0 0 24 Mar 2022
Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features N. Kalibhat Kanika Narang Hamed Firooz Maziar Sanjabi S. Feizi SSL 31 7 0 03 Mar 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 21 109 0 28 Feb 2022
A Self-Supervised Descriptor for Image Copy Detection Ed Pizzi Sreya . Dutta Roy Sugosh Nagavara Ravindra Priya Goyal Matthijs Douze SSL 34 116 0 21 Feb 2022
When, where, and how to add new neurons to ANNs Kaitlin Maile Emmanuel Rachelson H. Luga Dennis G. Wilson 19 14 0 17 Feb 2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language Alexei Baevski Wei-Ning Hsu Qiantong Xu Arun Babu Jiatao Gu Michael Auli SSL VLM ViT 35 835 0 07 Feb 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 52 34 0 29 Jan 2022
Robust Contrastive Learning against Noisy Views Ching-Yao Chuang R. Devon Hjelm Xin Eric Wang Vibhav Vineet Neel Joshi Antonio Torralba Stefanie Jegelka Ya-heng Song NoLa 13 68 0 12 Jan 2022
Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent Yi Li Yang Sun S. M. Naqvi 16 1 0 21 Dec 2021
Weakly-supervised Generative Adversarial Networks for medical image classification Jia-min Mao Xuesong Yin Yuan Chang Qi Huang GAN MedIm 19 1 0 29 Nov 2021
CoReS: Compatible Representations via Stationarity Niccoló Biondi F. Pernici Matteo Bruni A. Bimbo OOD 16 9 0 15 Nov 2021
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP Andreas Fürst Elisabeth Rumetshofer Johannes Lehner Viet-Hung Tran Fei Tang ... David P. Kreil Michael K Kopp G. Klambauer Angela Bitto-Nemling Sepp Hochreiter VLM CLIP 204 102 0 21 Oct 2021
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Y. Zou Linjun Zhang SSL 51 48 0 06 Oct 2021
Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation Chenyu You Ruihan Zhao Lawrence H. Staib James S. Duncan SSL 35 106 0 14 May 2021
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 124 187 0 02 May 2021
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations Debidatta Dwibedi Y. Aytar Jonathan Tompson P. Sermanet Andrew Zisserman SSL 188 452 0 29 Apr 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 314 5,775 0 29 Apr 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 138 279 0 12 Feb 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 267 3,369 0 09 Mar 2020