Understanding Contrastive Learning Requires Incorporating Inductive
Biases

Understanding Contrastive Learning Requires Incorporating Inductive Biases

28 February 2022

Dipendra Kumar Misra

A. Krishnamurthy

Papers citing "Understanding Contrastive Learning Requires Incorporating Inductive Biases"

13 / 13 papers shown

Title
Your contrastive learning problem is secretly a distribution alignment problem Zihao Chen Chi-Heng Lin Ran Liu Jingyun Xiao Eva L. Dyer 58 1 0 27 Feb 2025
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 33 5 0 28 Jun 2024
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models Dennis Wu Jerry Yao-Chieh Hu Teng-Yun Hsiao Han Liu 38 28 0 04 Apr 2024
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective Yue Xing Xiaofeng Lin Qifan Song Yi Tian Xu Belinda Zeng Guang Cheng SSL 8 0 0 26 Jan 2024
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 6 6 0 06 Jun 2023
GEDI: GEnerative and DIscriminative Training for Self-Supervised Learning Emanuele Sansone Robin Manhaeve SSL 13 9 0 27 Dec 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 21 7 0 25 Oct 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 13 34 0 12 May 2022
Do More Negative Samples Necessarily Hurt in Contrastive Learning? Pranjal Awasthi Nishanth Dikkala Pritish Kamath 15 40 0 03 May 2022
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,554 0 04 May 2021
Contrastive Learning Inverts the Data Generating Process Roland S. Zimmermann Yash Sharma Steffen Schneider Matthias Bethge Wieland Brendel SSL 230 206 0 17 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 114 201 0 16 Feb 2021
For self-supervised learning, Rationality implies generalization, provably Yamini Bansal Gal Kaplun Boaz Barak OOD SSL 47 22 0 16 Oct 2020