An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 389 papers shown

Title
Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review Yining Shi Kun Jiang Jiusi Li Zelin Qian Jun Wen Mengmeng Yang Ke Wang Diange Yang 78 25 0 02 Mar 2023
Applying Plain Transformers to Real-World Point Clouds Lanxiao Li M. Heizmann 3DPC ViT 23 3 0 28 Feb 2023
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking Chang-Shu Liu Yinpeng Dong Wenzhao Xiang X. Yang Hang Su Junyi Zhu YueFeng Chen Yuan He H. Xue Shibao Zheng OOD VLM AAML 24 72 0 28 Feb 2023
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations Ziyu Jiang Yinpeng Chen Mengchen Liu Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Zhangyang Wang SSL VLM CLIP 32 16 0 27 Feb 2023
Amortised Invariance Learning for Contrastive Self-Supervision Ruchika Chavhan H. Gouk Jan Stuehmer Calum Heggan Mehrdad Yaghoobi Timothy M. Hospedales SSL 32 11 0 24 Feb 2023
ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms Minzhou Pan Yi Zeng Lingjuan Lyu X. Lin R. Jia AAML 24 35 0 22 Feb 2023
Steerable Equivariant Representation Learning Sangnie Bhardwaj Willie McClinton Tongzhou Wang Guillaume Lajoie Chen Sun Phillip Isola Dilip Krishnan OOD LLMSV 26 5 0 22 Feb 2023
Self-supervised learning of Split Invariant Equivariant representations Q. Garrido Laurent Najman Yann LeCun SSL 24 32 0 14 Feb 2023
LipLearner: Customizable Silent Speech Interactions on Mobile Devices Zixiong Su Shitao Fang Jun Rekimoto 16 26 0 12 Feb 2023
Key Design Choices for Double-Transfer in Source-Free Unsupervised Domain Adaptation Andrea Maracani Raffaello Camoriano Elisa Maiettini Davide Talon Lorenzo Rosasco Lorenzo Natale 21 2 0 10 Feb 2023
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images Wei-Chien Wang E. Ahn Da-wei Feng Jinman Kim MedIm 21 27 0 10 Feb 2023
SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation Yash J. Patel Yusheng Xie Yi Zhu Srikar Appalaraju R. Manmatha 27 4 0 07 Feb 2023
AIM: Adapting Image Models for Efficient Video Action Recognition Taojiannan Yang Yi Zhu Yusheng Xie Aston Zhang C. L. P. Chen Mu Li ViT 44 144 0 06 Feb 2023
Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining Zekun Qi Runpei Dong Guo Fan Zheng Ge Xiangyu Zhang Kaisheng Ma Li Yi 32 117 0 05 Feb 2023
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval Shunyu Zhang Yaobo Liang Ming Gong Daxin Jiang Nan Duan 21 4 0 03 Feb 2023
Energy-Inspired Self-Supervised Pretraining for Vision Models Ze Wang Jiang Wang Zicheng Liu Qiang Qiu 21 8 0 02 Feb 2023
Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense Zunzhi You Daochang Liu Bohyung Han Chang Xu AAML VLM 41 4 0 02 Feb 2023
A Closer Look at Few-shot Classification Again Xu Luo Hao Wu Ji Zhang Lianli Gao Jing Xu Jingkuan Song 24 48 0 28 Jan 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wenjia Wang 17 8 0 28 Jan 2023
Self-Supervised Curricular Deep Learning for Chest X-Ray Image Classification Iván de Andrés Tamé Kirill Sirotkin Pablo Carballeira Marcos Escudero-Viñolo 21 2 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
ViT-AE++: Improving Vision Transformer Autoencoder for Self-supervised Medical Image Representations Chinmay Prabhakar Hongwei Bran Li Jiancheng Yang Suprosana Shit Benedikt Wiestler Bjoern H. Menze ViT MedIm 23 11 0 18 Jan 2023
Vision Learners Meet Web Image-Text Pairs Bingchen Zhao Quan Cui Hao Wu Osamu Yoshie Cheng Yang Oisin Mac Aodha VLM 24 5 0 17 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 28 11 0 11 Jan 2023
Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling Keyu Tian Yi-Xin Jiang Qishuai Diao Chen Lin Liwei Wang Zehuan Yuan 27 100 0 09 Jan 2023
Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning Jidong Ge YuXiang Liu Jie Gui Lanting Fang Ming Lin James T. Kwok LiGuo Huang B. Luo SSL 14 5 0 08 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 27 24 0 05 Jan 2023
Event Camera Data Pre-training Yan Yang Liyuan Pan Liu Liu 15 31 0 05 Jan 2023
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform Yutaro Shigeto Masashi Shimbo Yuya Yoshikawa A. Takeuchi 11 0 0 04 Jan 2023
Semi-MAE: Masked Autoencoders for Semi-supervised Vision Transformers Haojie Yu Kangnian Zhao Xiaoming Xu ViT 28 1 0 04 Jan 2023
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models Sucheng Ren Fangyun Wei Zheng-Wei Zhang Han Hu 35 34 0 03 Jan 2023
A New Perspective to Boost Vision Transformer for Medical Image Classification Yuexiang Li Yawen Huang Nanjun He Kai Ma Yefeng Zheng ViT MedIm 21 3 0 03 Jan 2023
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders Sanghyun Woo Shoubhik Debnath Ronghang Hu Xinlei Chen Zhuang Liu In So Kweon Saining Xie SyDa 76 723 0 02 Jan 2023
Precise Location Matching Improves Dense Contrastive Learning in Digital Pathology Jingwei Zhang S. Kapse Ke Ma Prateek Prasanna Maria Vakalopoulou Joel H. Saltz Dimitris Samaras 24 9 0 23 Dec 2022
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault SSL 14 6 0 21 Dec 2022
Image Segmentation-based Unsupervised Multiple Objects Discovery Sandra Kara Hejer Ammar Florian Chabot Q. C. Pham OCL 14 6 0 20 Dec 2022
Boosting Semi-Supervised Learning with Contrastive Complementary Labeling Qinyi Deng Yong Guo Zhibang Yang Haolin Pan Jian Chen 22 10 0 13 Dec 2022
FastMIM: Expediting Masked Image Modeling Pre-training for Vision Jianyuan Guo Kai Han Han Wu Yehui Tang Yunhe Wang Chang Xu 27 9 0 13 Dec 2022
Learning Imbalanced Data with Vision Transformers Zhengzhuo Xu R. Liu Shuo Yang Zenghao Chai Chun Yuan 35 32 0 05 Dec 2022
Exploring Stochastic Autoregressive Image Modeling for Visual Representation Yu-Hang Qi Fan Yang Yousong Zhu Yufei Liu Liwei Wu Rui Zhao Wei Li DiffM 27 13 0 03 Dec 2022
Finetune like you pretrain: Improved finetuning of zero-shot vision models Sachin Goyal Ananya Kumar Sankalp Garg Zico Kolter Aditi Raghunathan CLIP VLM 31 136 0 01 Dec 2022
Exploiting Category Names for Few-Shot Classification with Vision-Language Models Taihong Xiao Zirui Wang Liangliang Cao Jiahui Yu Shengyang Dai Ming Yang VLM MLLM 27 5 0 29 Nov 2022
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation Jiangyong Huang William Zhu Baoxiong Jia Zan Wang Xiaojian Ma Qing Li Siyuan Huang 32 5 0 28 Nov 2022
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix Wenbin Li Meihao Kong Xuesong Yang Lei Wang Jing Huo Yang Gao Jiebo Luo 25 0 0 26 Nov 2022
Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation Liang Zeng A. Lengyel Nergis Tomen J. C. V. Gemert AI4TS 19 0 0 25 Nov 2022
Self-Supervised Learning based on Heat Equation Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 29 4 0 23 Nov 2022
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Peng Jin Jinfa Huang Fenglin Liu Xian Wu Shen Ge Guoli Song David A. Clifton Jing Chen VLM 34 63 0 21 Nov 2022
Cross-Modal Contrastive Learning for Robust Reasoning in VQA Qinjie Zheng Chaoyue Wang Daqing Liu Dadong Wang Dacheng Tao LRM 21 0 0 21 Nov 2022
Explanation on Pretraining Bias of Finetuned Vision Transformer Bumjin Park Jaesik Choi ViT 29 1 0 18 Nov 2022
Self-Supervised Visual Representation Learning via Residual Momentum T. Pham Axi Niu Zhang Kang Sultan Rizky Hikmawan Madjid Jiajing Hong Daehyeok Kim Joshua Tian Jin Tee Chang-Dong Yoo SSL 41 6 0 17 Nov 2022