Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning

31 May 2021

Papers citing "Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning"

28 / 28 papers shown

Title
A Probabilistic Model for Self-Supervised Learning Maximilian Fleissner P. Esser D. Ghoshdastidar SSL BDL 93 1 0 22 Jan 2025
Towards Understanding Why FixMatch Generalizes Better Than Supervised Learning Jingyang Li Jiachun Pan Vincent Y. F. Tan Kim-Chuan Toh Pan Zhou AAML MLT 43 0 0 15 Oct 2024
Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data Binghui Li Yuanzhi Li OOD 28 2 0 11 Oct 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 33 5 0 28 Jun 2024
IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning Prashant Bhat Bharath Renjith Elahe Arani Bahram Zonooz CLL 40 2 0 28 Apr 2024
Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlations GuanWen Qiu Da Kuang Surbhi Goel 25 8 0 05 Mar 2024
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective Yue Xing Xiaofeng Lin Qifan Song Yi Tian Xu Belinda Zeng Guang Cheng SSL 15 0 0 26 Jan 2024
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 19 6 0 06 Jun 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 29 29 0 27 Mar 2023
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity Hongkang Li M. Wang Sijia Liu Pin-Yu Chen ViT MLT 35 56 0 12 Feb 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 11 31 0 06 Feb 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wenjia Wang 17 8 0 28 Jan 2023
Augmentation Invariant Manifold Learning Shulei Wang 29 1 0 01 Nov 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 41 8 0 25 Oct 2022
A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning Guozheng Ma Zhen Wang Zhecheng Yuan Xueqian Wang Bo Yuan Dacheng Tao OffRL 30 26 0 10 Oct 2022
Joint Embedding Self-Supervised Learning in the Kernel Regime B. Kiani Randall Balestriero Yubei Chen S. Lloyd Yann LeCun SSL 41 13 0 29 Sep 2022
Towards Understanding Mixture of Experts in Deep Learning Zixiang Chen Yihe Deng Yue-bo Wu Quanquan Gu Yuan-Fang Li MLT MoE 27 53 0 04 Aug 2022
Learn2Weight: Parameter Adaptation against Similar-domain Adversarial Attacks Siddhartha Datta AAML 30 4 0 15 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 27 34 0 12 May 2022
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey Kento Nozawa Issei Sato AI4TS 16 4 0 18 Apr 2022
Frequency Selective Augmentation for Video Representation Learning Jinhyung Kim Taeoh Kim Minho Shim Dongyoon Han Dongyoon Wee Junmo Kim AI4TS 41 3 0 08 Apr 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 19 109 0 28 Feb 2022
Sample Efficiency of Data Augmentation Consistency Regularization Shuo Yang Yijun Dong Rachel A. Ward Inderjit S. Dhillon Sujay Sanghavi Qi Lei AAML 23 17 0 24 Feb 2022
Towards the Generalization of Contrastive Self-Supervised Learning Weiran Huang Mingyang Yi Xuyang Zhao Zihao Jiang SSL 21 105 0 01 Nov 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision Xiang Wang Xinlei Chen S. Du Yuandong Tian SSL 16 26 0 11 Oct 2021
On the Surrogate Gap between Contrastive and Supervised Losses Han Bao Yoshihiro Nagano Kento Nozawa SSL UQCV 37 19 0 06 Oct 2021
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Y. Zou Linjun Zhang SSL 51 48 0 06 Oct 2021
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 230 31,253 0 16 Jan 2013