Efficient Training of Visual Transformers with Small Datasets

7 June 2021

Wei Bi

Papers citing "Efficient Training of Visual Transformers with Small Datasets"

29 / 29 papers shown

Title
SONICS: Synthetic Or Not -- Identifying Counterfeit Songs Md Awsafur Rahman Zaber Ibn Abdul Hakim Najibul Haque Sarker Bishmoy Paul S. Fattah 38 7 0 26 Aug 2024
TULIP: Transformer for Upsampling of LiDAR Point Clouds Bin Yang Patrick Pfreundschuh Roland Siegwart Marco Hutter Peyman Moghadam Vaishakh Patil 3DPC 18 5 0 11 Dec 2023
Are Vision Transformers More Data Hungry Than Newborn Visual Systems? Lalit Pandey Samantha M. W. Wood Justin N. Wood 31 12 0 05 Dec 2023
A Deep Learning Approach to Teeth Segmentation and Orientation from Panoramic X-rays M. K. Dhar Mou Deb D. Madhab Zeyun Yu 17 1 0 26 Oct 2023
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang F. Khan ViT 35 84 0 27 Mar 2023
Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network Yudong Li Yunlin Lei Xu Yang 21 26 0 19 Nov 2022
Training a Vision Transformer from scratch in less than 24 hours with 1 GPU Saghar Irandoust Thibaut Durand Yunduz Rakhmangulova Wenjie Zi Hossein Hajimirsadeghi ViT 33 6 0 09 Nov 2022
Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets Xiangyu Chen Ying Qin Wenju Xu A. Bur Cuncong Zhong Guanghui Wang ViT 38 3 0 25 Oct 2022
Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Zhiying Lu Hongtao Xie Chuanbin Liu Yongdong Zhang ViT 10 57 0 12 Oct 2022
Relaxed Attention for Transformer Models Timo Lohrenz Björn Möller Zhengyang Li Tim Fingscheidt KELM 24 11 0 20 Sep 2022
How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classification Vincent Tonkes M. Sabatelli ViT 25 6 0 09 Aug 2022
Locality Guidance for Improving Vision Transformers on Tiny Datasets Kehan Li Runyi Yu Zhennan Wang Li-ming Yuan Guoli Song Jie Chen ViT 24 43 0 20 Jul 2022
Rethinking Generalization in Few-Shot Classification Markus Hiller Rongkai Ma Mehrtash Harandi Tom Drummond OCL VLM 17 55 0 15 Jun 2022
Few-Shot Diffusion Models Giorgio Giannone Didrik Nielsen Ole Winther DiffM 178 49 0 30 May 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 110 17 0 30 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 25 180 0 06 May 2022
Representation Learning by Detecting Incorrect Location Embeddings Sepehr Sameni Simon Jenni Paolo Favaro ViT 23 4 0 10 Apr 2022
Towards Data-Efficient Detection Transformers Wen Wang Jing Zhang Yang Cao Yongliang Shen Dacheng Tao ViT 18 59 0 17 Mar 2022
Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs Jingfei Xia Mingchen Zhuge Tiantian Geng Shun Fan Yuantai Wei Zhenyu He Feng Zheng 21 14 0 08 Mar 2022
Exploiting Spatial Sparsity for Event Cameras with Visual Transformers Zuowen Wang Yuhuang Hu Shih-Chii Liu ViT 22 33 0 10 Feb 2022
Are Large-scale Datasets Necessary for Self-Supervised Pre-training? Alaaeldin El-Nouby Gautier Izacard Hugo Touvron Ivan Laptev Hervé Jégou Edouard Grave SSL 13 148 0 20 Dec 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 15 2,744 0 15 Jun 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 251 620 0 21 May 2021
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 114 187 0 02 May 2021
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations Debidatta Dwibedi Y. Aytar Jonathan Tompson P. Sermanet Andrew Zisserman SSL 183 452 0 29 Apr 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 303 5,773 0 29 Apr 2021
TrackFormer: Multi-Object Tracking with Transformers Tim Meinhardt A. Kirillov Laura Leal-Taixe Christoph Feichtenhofer VOT 218 742 0 07 Jan 2021
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 253 1,824 0 18 Aug 2016
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 228 31,244 0 16 Jan 2013