An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 389 papers shown

Title
Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer Yingyi Chen Xiaoke Shen Yahui Liu Qinghua Tao Johan A. K. Suykens AAML ViT 21 22 0 25 Jul 2022
Improved Super Resolution of MR Images Using CNNs and Vision Transformers Dwarikanath Mahapatra SupR ViT MedIm 19 5 0 24 Jul 2022
High-Resolution Swin Transformer for Automatic Medical Image Segmentation Chen Wei Shenghan Ren Kaitai Guo Haihong Hu Jimin Liang ViT OOD MedIm 17 36 0 23 Jul 2022
Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility Rohit Gupta Naveed Akhtar Ajmal Saeed Mian M. Shah AAML SSL 21 5 0 22 Jul 2022
Negative Samples are at Large: Leveraging Hard-distance Elastic Loss for Re-identification Hyungtae Lee Sungmin Eum H. Kwon VLM 15 4 0 20 Jul 2022
Position Prediction as an Effective Pretraining Strategy Shuangfei Zhai Navdeep Jaitly Jason Ramapuram Dan Busbridge Tatiana Likhomanenko Joseph Y. Cheng Walter A. Talbott Chen Huang Hanlin Goh J. Susskind ViT 35 23 0 15 Jul 2022
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Yiwei Ma Guohai Xu Xiaoshuai Sun Ming Yan Ji Zhang Rongrong Ji CLIP VLM 10 267 0 15 Jul 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 13 268 0 13 Jul 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 18 35 0 13 Jul 2022
Facilitated machine learning for image-based fruit quality assessment Manuel Knott F. Pérez-Cruz T. Defraeye 14 47 0 10 Jul 2022
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition Wenhao Wu Zhun Sun Wanli Ouyang VLM 99 93 0 04 Jul 2022
Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognition Mingkun Yang Minghui Liao Pu Lu Jing Wang Shenggao Zhu Hualin Luo Qingzhen Tian X. Bai SSL 29 55 0 01 Jul 2022
Masked World Models for Visual Control Younggyo Seo Danijar Hafner Hao Liu Fangchen Liu Stephen James Kimin Lee Pieter Abbeel OffRL 77 145 0 28 Jun 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 30 32 0 19 Jun 2022
Bootstrapped Transformer for Offline Reinforcement Learning Kerong Wang Hanye Zhao Xufang Luo Kan Ren Weinan Zhang Dongsheng Li OffRL 16 37 0 17 Jun 2022
Rectify ViT Shortcut Learning by Visual Saliency Chong Ma Lin Zhao Yuzhong Chen David Liu Xi Jiang Tuo Zhang Xintao Hu Dinggang Shen Dajiang Zhu Tianming Liu ViT 25 20 0 17 Jun 2022
OmniMAE: Single Model Masked Pretraining on Images and Videos Rohit Girdhar Alaaeldin El-Nouby Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra ViT 32 97 0 16 Jun 2022
Evaluating Self-Supervised Learning for Molecular Graph Embeddings Hanchen Wang Jean Kaddour Shengchao Liu Jian Tang Joan Lasenby Qi Liu 24 20 0 16 Jun 2022
Patch-level Representation Learning for Self-supervised Vision Transformers Sukmin Yun Hankook Lee Jaehyung Kim Jinwoo Shin ViT 22 64 0 16 Jun 2022
Masked Siamese ConvNets L. Jing Jiachen Zhu Yann LeCun SSL 35 34 0 15 Jun 2022
Rethinking Generalization in Few-Shot Classification Markus Hiller Rongkai Ma Mehrtash Harandi Tom Drummond OCL VLM 17 55 0 15 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 47 525 0 13 Jun 2022
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation Wouter Van Gansbeke Simon Vandenhende Luc Van Gool 34 55 0 13 Jun 2022
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels? Xiang Li Jinghuan Shang Srijan Das Michael S. Ryoo SSL 25 31 0 10 Jun 2022
Extreme Masking for Learning Instance and Distributed Visual Representations Zhirong Wu Zihang Lai Xiao Sun Stephen Lin 30 22 0 09 Jun 2022
Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction Jun Chen Ming Hu Boyang Albert Li Mohamed Elhoseiny 37 36 0 01 Jun 2022
Contrastive Centroid Supervision Alleviates Domain Shift in Medical Image Classification Wenshuo Zhou Dalu Yang Binghong Wu Yehui Yang Junde Wu Xiaorong Wang Lei Wang Haifeng Huang Yanwu Xu OOD 39 1 0 31 May 2022
Self-Supervised Pre-training of Vision Transformers for Dense Prediction Tasks Jaonary Rabarisoa Velentin Belissen Florian Chabot Q. C. Pham VLM ViT SSL MDE 13 2 0 30 May 2022
GMML is All you Need Sara Atito Muhammad Awais J. Kittler ViT VLM 44 18 0 30 May 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 52 26 0 30 May 2022
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners Feng Liang Yangguang Li Diana Marculescu SSL TPM ViT 49 22 0 28 May 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian-jun Sun Weiming Hu ViT 62 41 0 28 May 2022
Architecture-Agnostic Masked Image Modeling -- From ViT back to CNN Siyuan Li Di Wu Fang Wu Lei Shang Stan.Z.Li 32 48 0 27 May 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 24 68 0 26 May 2022
Improvements to Self-Supervised Representation Learning for Masked Image Modeling Jia-ju Mao Xuesong Yin Yuan Chang Honggu Zhou SSL 24 1 0 21 May 2022
Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT) Jue Jiang N. Tyagi K. Tringale C. Crane H. Veeraraghavan MedIm 33 34 0 20 May 2022
Masked Image Modeling with Denoising Contrast Kun Yi Yixiao Ge Xiaotong Li Shusheng Yang Dian Li Jianping Wu Ying Shan Xiaohu Qie VLM 30 51 0 19 May 2022
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization Luke Melas-Kyriazi Christian Rupprecht Iro Laina Andrea Vedaldi 28 159 0 16 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 24 34 0 12 May 2022
Scene Consistency Representation Learning for Video Scene Segmentation Haoqian Wu Keyu Chen Yanan Luo Ruizhi Qiao Bo Ren Haozhe Liu Weicheng Xie Linlin Shen SSL 31 16 0 11 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 32 33 0 10 May 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao Teli Ma Hongsheng Li Ziyi Lin Jifeng Dai Yu Qiao ViT 19 121 0 08 May 2022
Relational Representation Learning in Visually-Rich Documents Xin Li Yan Zheng Yiqing Hu H. Cao Yunfei Wu Deqiang Jiang Yinsong Liu Bo Ren 18 12 0 05 May 2022
Better plain ViT baselines for ImageNet-1k Lucas Beyer Xiaohua Zhai Alexander Kolesnikov ViT VLM 27 111 0 03 May 2022
RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning Xiaojian Ma Weili Nie Zhiding Yu Huaizu Jiang Chaowei Xiao Yuke Zhu Song-Chun Zhu Anima Anandkumar ViT LRM 22 19 0 24 Apr 2022
Neuro-BERT: Rethinking Masked Autoencoding for Self-supervised Neurological Pretraining Di Wu Siyuan Li Jie Yang Mohamad Sawan SSL 28 14 0 20 Apr 2022
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models Chunyuan Li Haotian Liu Liunian Harold Li Pengchuan Zhang J. Aneja ... Ping Jin Houdong Hu Zicheng Liu Yong Jae Lee Jianfeng Gao 26 144 0 19 Apr 2022
Masked Siamese Networks for Label-Efficient Learning Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Florian Bordes Pascal Vincent Armand Joulin Michael G. Rabbat Nicolas Ballas SSL 16 311 0 14 Apr 2022
Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixels Tianxin Tao Daniele Reda M. van de Panne ViT 11 19 0 11 Apr 2022
Representation Learning by Detecting Incorrect Location Embeddings Sepehr Sameni Simon Jenni Paolo Favaro ViT 29 4 0 10 Apr 2022