How to train your ViT? Data, Augmentation, and Regularization in Vision
Transformers

How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

18 June 2021

Andreas Steiner

Alexander Kolesnikov

Jakob Uszkoreit

Papers citing "How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers"

15 / 415 papers shown

Title
The Benchmark Lottery Mostafa Dehghani Yi Tay A. Gritsenko Zhe Zhao N. Houlsby Fernando Diaz Donald Metzler Oriol Vinyals 10 88 0 14 Jul 2021
The Brownian motion in the transformer model Yingshi Chen 14 1 0 12 Jul 2021
Exploring the Limits of Out-of-Distribution Detection Stanislav Fort Jie Jessie Ren Balaji Lakshminarayanan 11 322 0 06 Jun 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 17 300 0 17 May 2021
Segmenter: Transformer for Semantic Segmentation Robin Strudel Ricardo Garcia Pinel Ivan Laptev Cordelia Schmid ViT 21 1,410 0 12 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,554 0 04 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 5,723 0 29 Apr 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 109 206 0 26 Apr 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 160 676 0 22 Apr 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 11 197 0 08 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 1,077 0 17 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Tsung-Yi Lin Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 267 955 0 27 Jan 2021
Fine-grained Species Recognition with Privileged Pooling: Better Sample Efficiency Through Supervised Attention Andrés C. Rodríguez Stefano Dáronco Konrad Schindler Jan Dirk Wegner 11 4 0 20 Mar 2020