Beyond Masking: Demystifying Token-Based Pre-Training for Vision
Transformers

Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers

27 March 2022

Papers citing "Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers"

9 / 9 papers shown

Title
Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense Zunzhi You Daochang Liu Bohyung Han Chang Xu AAML VLM 24 4 0 02 Feb 2023
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond Chaoning Zhang Chenshuang Zhang Junha Song John Seon Keun Yi Kang Zhang In So Kweon SSL 39 70 0 30 Jul 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 50 26 0 30 May 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 189 1,148 0 05 Oct 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 5,723 0 29 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 238 3,359 0 09 Mar 2020
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 279 39,083 0 01 Sep 2014