Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage

6 June 2023

Papers citing "Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage"

10 / 10 papers shown

Title
Projection Head is Secretly an Information Bottleneck Zhuo Ouyang Kaiwen Hu Qi Zhang Yifei Wang Yisen Wang 37 0 0 01 Mar 2025
Improving Nonlinear Projection Heads using Pretrained Autoencoder Embeddings Andreas Schliebitz Heiko Tapken Martin Atzmueller 27 0 0 25 Aug 2024
DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations Guogang Zhu Xuefeng Liu Jianwei Niu Shaojie Tang Xinghao Wu Jiayuan Zhang AI4CE 45 0 0 25 Jul 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 26 10 0 18 Mar 2024
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 155 65 0 27 Oct 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 46 34 0 29 Jan 2022
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 107 163 0 02 May 2021
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations Debidatta Dwibedi Y. Aytar Jonathan Tompson P. Sermanet Andrew Zisserman SSL 183 450 0 29 Apr 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 132 278 0 12 Feb 2021
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 347 0 14 Jun 2018