Predicting What You Already Know Helps: Provable Self-Supervised Learning

3 August 2020

Papers citing "Predicting What You Already Know Helps: Provable Self-Supervised Learning"

43 / 43 papers shown

Title
CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoders Jongwon Park Heesoo Jung Hogun Park 51 1 0 10 Mar 2025
THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings Bowen Deng Tong Wang Lele Fu Sheng Huang Chuan Chen Tao Zhang 82 0 0 17 Feb 2025
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 34 0 0 01 Oct 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 33 5 0 28 Jun 2024
A Survey on Self-Supervised Learning for Non-Sequential Tabular Data Wei-Yao Wang Wei-Wei Du Derek Xu Wei Wang Wenjie Peng LMTD 32 7 0 02 Feb 2024
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
How does Contrastive Learning Organize Images? Yunzhe Zhang Yao Lu Qi Xuan SSL 26 0 0 17 May 2023
On the Provable Advantage of Unsupervised Pretraining Jiawei Ge Shange Tang Jianqing Fan Chi Jin SSL 33 16 0 02 Mar 2023
Hiding Data Helps: On the Benefits of Masking for Sparse Coding Muthuraman Chidambaram Chenwei Wu Yu Cheng Rong Ge 18 0 0 24 Feb 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 11 31 0 06 Feb 2023
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications Chenyu Zheng Guoqiang Wu Fan Bao Yue Cao Chongxuan Li Jun Zhu BDL 25 30 0 05 Feb 2023
Homomorphic Self-Supervised Learning Thomas Anderson Keller Xavier Suau Luca Zappella SSL 19 2 0 15 Nov 2022
Joint Embedding Self-Supervised Learning in the Kernel Regime B. Kiani Randall Balestriero Yubei Chen S. Lloyd Yann LeCun SSL 41 13 0 29 Sep 2022
Efficient Medical Image Assessment via Self-supervised Learning Chun-Yin Huang Qi Lei Xiaoxiao Li 17 2 0 28 Sep 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 27 34 0 12 May 2022
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey Kento Nozawa Issei Sato AI4TS 19 4 0 18 Apr 2022
Augmentation-Free Graph Contrastive Learning with Performance Guarantee Haonan Wang Jieyu Zhang Qi Zhu Wei Huang 30 31 0 11 Apr 2022
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap Yifei Wang Qi Zhang Yisen Wang Jiansheng Yang Zhouchen Lin 21 98 0 25 Mar 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 35 106 0 02 Mar 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 21 109 0 28 Feb 2022
Self-Training: A Survey Massih-Reza Amini Vasilii Feofanov Loïc Pauletto Lies Hadjadj Emilie Devijver Yury Maximov SSL 28 102 0 24 Feb 2022
Conditional Contrastive Learning with Kernel Yao-Hung Hubert Tsai Tianqi Li Martin Q. Ma Han Zhao Kun Zhang Louis-Philippe Morency Ruslan Salakhutdinov 19 25 0 11 Feb 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 52 34 0 29 Jan 2022
Non-Stationary Representation Learning in Sequential Linear Bandits Yuzhen Qin Tommaso Menara Samet Oymak ShiNung Ching Fabio Pasqualetti OffRL 32 17 0 13 Jan 2022
Towards the Generalization of Contrastive Self-Supervised Learning Weiran Huang Mingyang Yi Xuyang Zhao Zihao Jiang SSL 21 105 0 01 Nov 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 25 336 0 18 Oct 2021
Self-Supervised Representation Learning: Introduction, Advances and Challenges Linus Ericsson H. Gouk Chen Change Loy Timothy M. Hospedales SSL OOD AI4TS 29 271 0 18 Oct 2021
Self-supervised Learning is More Robust to Dataset Imbalance Hong Liu Jeff Z. HaoChen Adrien Gaidon Tengyu Ma OOD SSL 31 157 0 11 Oct 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision Xiang Wang Xinlei Chen S. Du Yuandong Tian SSL 18 26 0 11 Oct 2021
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Y. Zou Linjun Zhang SSL 51 48 0 06 Oct 2021
Can contrastive learning avoid shortcut solutions? Joshua Robinson Li Sun Ke Yu Kayhan Batmanghelich Stefanie Jegelka S. Sra SSL 19 141 0 21 Jun 2021
Investigating the Role of Negatives in Contrastive Representation Learning Jordan T. Ash Surbhi Goel A. Krishnamurthy Dipendra Kumar Misra SSL 29 49 0 18 Jun 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 22 96 0 17 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 24 131 0 31 May 2021
Distill on the Go: Online knowledge distillation in self-supervised learning Prashant Bhat Elahe Arani Bahram Zonooz SSL 22 28 0 20 Apr 2021
Can Pretext-Based Self-Supervised Learning Be Boosted by Downstream Data? A Theoretical Analysis Jiaye Teng Weiran Huang Haowei He SSL 29 11 0 05 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 29 8 0 02 Mar 2021
Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning Kento Nozawa Issei Sato SSL 20 43 0 13 Feb 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 138 279 0 12 Feb 2021
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 30 173 0 05 Nov 2020
For self-supervised learning, Rationality implies generalization, provably Yamini Bansal Gal Kaplun Boaz Barak OOD SSL 58 22 0 16 Oct 2020
Hard Negative Mixing for Contrastive Learning Yannis Kalantidis Mert Bulent Sariyildiz Noé Pion Philippe Weinzaepfel Diane Larlus SSL 29 628 0 02 Oct 2020
Understanding Self-supervised Learning with Dual Deep Networks Yuandong Tian Lantao Yu Xinlei Chen Surya Ganguli SSL 13 78 0 01 Oct 2020