The Power of Contrast for Feature Learning: A Theoretical Analysis

6 October 2021

Papers citing "The Power of Contrast for Feature Learning: A Theoretical Analysis"

39 / 39 papers shown

Title
DivIL: Unveiling and Addressing Over-Invariance for Out-of- Distribution Generalization Jiaqi Wang Yuhang Zhou Zhixiong Zhang Qiguang Chen Yongqiang Chen James Cheng OODD 53 1 0 18 Feb 2025
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features Po-han Li Sandeep P. Chinchali Ufuk Topcu 26 1 0 10 Oct 2024
Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learning Yunhui Liu Huaisong Zhang Tieke He Tao Zheng Jianhua Zhao SSL 24 1 0 09 Aug 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 21 0 0 28 May 2024
Contrastive Learning on Multimodal Analysis of Electronic Health Records Tianxi Cai Feiqing Huang Ryumei Nakada Linjun Zhang Doudou Zhou 28 0 0 22 Mar 2024
Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity Siddharth Joshi Arnav Jain Ali Payani Baharan Mirzasoleiman VLM CLIP 25 8 0 18 Mar 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 16 10 0 18 Mar 2024
When can we Approximate Wide Contrastive Models with Neural Tangent Kernels and Principal Component Analysis? Gautham Govind Anil P. Esser D. Ghoshdastidar 16 1 0 13 Mar 2024
Provable Multi-Party Reinforcement Learning with Diverse Human Feedback Huiying Zhong Zhun Deng Weijie J. Su Zhiwei Steven Wu Linjun Zhang 28 13 0 08 Mar 2024
Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning Yiping Wang Yifang Chen Wendan Yan Kevin G. Jamieson S. Du 13 5 0 03 Feb 2024
Towards Calibrated Robust Fine-Tuning of Vision-Language Models Changdae Oh Hyesu Lim Mijoo Kim Dongyoon Han Junhyeok Park Euiseog Jeong Alexander G. Hauptmann Zhi-Qi Cheng Kyungwoo Song VLM 8 13 0 03 Nov 2023
Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift Yihao Xue Siddharth Joshi Dang Nguyen Baharan Mirzasoleiman VLM 21 4 0 08 Oct 2023
Complexity Matters: Rethinking the Latent Space for Generative Modeling Tianyang Hu Fei Chen Hong Wang Jiawei Li Wenjia Wang Jiacheng Sun Z. Li DiffM 14 8 0 17 Jul 2023
Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training Alyssa Huang Peihan Liu Ryumei Nakada Linjun Zhang Wanrong Zhang VLM 12 5 0 13 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 6 6 0 06 Jun 2023
How Does Information Bottleneck Help Deep Learning? Kenji Kawaguchi Zhun Deng Xu Ji Jiaoyang Huang 21 46 0 30 May 2023
HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts Shaokun Zhang Yiran Wu Zhonghua Zheng Qingyun Wu Chi Wang OOD 25 7 0 28 May 2023
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression Yihao Xue S. Joshi Eric Gan Pin-Yu Chen Baharan Mirzasoleiman SSL 23 22 0 25 May 2023
Discover and Cure: Concept-aware Mitigation of Spurious Correlation Shirley Wu Mert Yuksekgonul Linjun Zhang James Y. Zou 57 54 0 01 May 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 31 270 0 24 Apr 2023
Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks Yuzhen Mao Zhun Deng Huaxiu Yao Ting Ye Kenji Kawaguchi James Y. Zou 19 18 0 08 Apr 2023
HappyMap: A Generalized Multi-calibration Method Zhun Deng Cynthia Dwork Linjun Zhang 63 17 0 08 Mar 2023
InfoNCE Loss Provably Learns Cluster-Preserving Representations Advait Parulekar Liam Collins Karthikeyan Shanmugam Aryan Mokhtari Sanjay Shakkottai SSL 19 12 0 15 Feb 2023
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data Ryumei Nakada Halil Ibrahim Gulluk Zhun Deng Wenlong Ji James Y. Zou Linjun Zhang SSL VLM 25 25 0 13 Feb 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wenjia Wang 13 5 0 28 Jan 2023
Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time Huaxiu Yao Caroline Choi Bochuan Cao Yoonho Lee Pang Wei Koh Chelsea Finn OOD 6 72 0 25 Nov 2022
Homomorphic Self-Supervised Learning Thomas Anderson Keller Xavier Suau Luca Zappella SSL 14 2 0 15 Nov 2022
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise Haotian Ye James Y. Zou Linjun Zhang OOD 14 20 0 20 Oct 2022
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data Zhun Deng Jiayao Zhang Linjun Zhang Ting Ye Yates Coley Weijie J. Su James Y. Zou 14 16 0 06 Jun 2022
Guided Deep Metric Learning Jorge Gonzalez-Zapata Iván Reyes-Amezcua Daniel Flores-Araiza M. Mendez-Ruiz G. Ochoa-Ruiz Andres Mendez-Vazquez FedML 12 5 0 04 Jun 2022
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning Yuandong Tian MLT 11 13 0 02 Jun 2022
Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding Tianyang Hu Zhili Liu Fengwei Zhou Wenjia Wang Weiran Huang SSL 23 18 0 30 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 13 34 0 12 May 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 43 26 0 29 Jan 2022
Scaffolding Sets M. Burhanpurkar Zhun Deng Cynthia Dwork Linjun Zhang 14 9 0 04 Nov 2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 114 65 0 06 Oct 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 130 258 0 12 Feb 2021
When and How Mixup Improves Calibration Linjun Zhang Zhun Deng Kenji Kawaguchi James Y. Zou UQCV 12 63 0 11 Feb 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 229 3,029 0 09 Mar 2020