Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations

27 February 2023

Lu Yuan

Zicheng Liu

Papers citing "Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations"

20 / 20 papers shown

Title
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization Siyuan Li L. Zhang Zedong Wang Juanxi Tian Cheng Tan ... Chang Yu Qingsong Xie Haonan Lu Haoqian Wang Zhen Lei 46 0 0 01 Apr 2025
Should we pre-train a decoder in contrastive learning for dense prediction tasks? S. Quetin Tapotosh Ghosh Farhad Maleki SSL 67 0 0 21 Mar 2025
MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations Benedikt Alkin Lukas Miklautz Sepp Hochreiter Johannes Brandstetter VLM 55 8 0 24 Feb 2025
PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings Sutharsan Mahendren Saimunur Rahman Piotr Koniusz Tharindu Fernando S. Sridharan Clinton Fookes Peyman Moghadam 3DPC 16 0 0 24 Sep 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 53 6 0 13 Aug 2024
Investigating Self-Supervised Methods for Label-Efficient Learning S. Nandam Sara Atito Zhenhua Feng Josef Kittler Muhammad Awais VLM 24 0 0 25 Jun 2024
Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision M. Taher Michael B. Gotway Jianming Liang MedIm 19 4 0 24 Apr 2024
CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification Guangqian Yang Kangrui Du Zhihan Yang Ye Du Yongping Zheng Shujun Wang 29 16 0 25 Mar 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 29 13 0 31 Dec 2023
Learning to Embed Time Series Patches Independently Seunghan Lee Taeyoung Park Kibok Lee SSL AI4TS 11 26 0 27 Dec 2023
Pre-training with Random Orthogonal Projection Image Modeling Maryam Haghighat Peyman Moghadam Shaheer Mohamed Piotr Koniusz VLM 8 7 0 28 Oct 2023
Adaptive Multi-head Contrastive Learning Lei Wang Piotr Koniusz Tom Gedeon Liang Zheng 26 4 0 09 Oct 2023
Masked Image Residual Learning for Scaling Deeper Vision Transformers Guoxi Huang Hongtao Fu A. Bors 11 7 0 25 Sep 2023
Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget Johannes Lehner Benedikt Alkin Andreas Fürst Elisabeth Rumetshofer Lukas Miklautz Sepp Hochreiter 11 18 0 20 Apr 2023
MimCo: Masked Image Modeling Pre-training with Contrastive Teacher Qiang-feng Zhou Chaohui Yu Haowen Luo Zhibin Wang Hao Li VLM 51 20 0 07 Sep 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian-jun Sun Weiming Hu ViT 60 40 0 28 May 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations Debidatta Dwibedi Y. Aytar Jonathan Tompson P. Sermanet Andrew Zisserman SSL 183 450 0 29 Apr 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 5,723 0 29 Apr 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 235 3,029 0 09 Mar 2020