How to Train Vision Transformer on Small-scale Datasets?

British Machine Vision Conference (BMVC), 2022

13 October 2022

ArXiv (abs)PDF HTML Github (154★)

Papers citing "How to Train Vision Transformer on Small-scale Datasets?"

32 / 32 papers shown

Title
Growing Winning Subnetworks, Not Pruning Them: A Paradigm for Density Discovery in Sparse Neural Networks Qihang Yao Constantine Dovrolis 112 0 0 30 Sep 2025
CoSwin: Convolution Enhanced Hierarchical Shifted Window Attention For Small-Scale Vision Puskal Khadka Rodrigue Rizk Longwei Wang KC Santosh ViT 97 0 0 10 Sep 2025
Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis Ethan Dack Lorenzo Brigato Vasilis Dedousis Janine Gote-Schniering Cheryl ... M. Funke-Chambour Thomas Geiser Andreas Christe L. Ebner Stavroula Mougiakakou 94 0 0 06 Aug 2025
Learning to Adapt to Position Bias in Vision Transformer Classifiers Robert-Jan Bruintjes Jan van Gemert 341 0 0 19 May 2025
Powerful Design of Small Vision Transformer on CIFAR10 Gent Wu ViT 240 1 0 07 Jan 2025
Cyborg Insect Factory: Automatic Assembly System to Build up Insect-computer Hybrid Robot Based on Vision-guided Robotic Arm Manipulation of Custom Bipolar Electrodes Qifeng Lin Nghia Vuong Kewei Song Phuoc Thanh Tran Ngoc Greg Angelo Gonzales Nonato Hirotaka Sato 175 0 0 20 Nov 2024
CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction Chunlei Meng Jiacheng Yang Wei Lin Bowen Liu Hongda Zhang chun ouyang Zhongxue Gan ViT 237 3 0 15 Oct 2024
An Enhanced Harmonic Densely Connected Hybrid Transformer Network Architecture for Chronic Wound Segmentation Utilising Multi-Colour Space Tensor Merging B. Cassidy Christian Mcbride Connah Kendrick N. Reeves Joseph M Pappachan ... Metib Alotaibi Abdullah Abdulaziz AlWabel Mohammad Alderwish Kuan-Ying Lai Moi Hoon Yap 237 4 0 04 Oct 2024
Y-CA-Net: A Convolutional Attention Based Network for Volumetric Medical Image Segmentation Muhammad Hamza Sharif Muzammal Naseer Mohammad Yaqub Min Xu Mohsen Guizani OOD MedIm ViT 141 1 0 01 Oct 2024
Self-supervised Auxiliary Learning for Texture and Model-based Hybrid Robust and Fair Featuring in Face AnalysisInternational Conference on Pattern Recognition (ICPR), 2024 Shukesh Reddy Nishit Poddar Srijan Das Abhijit Das CVBM 311 2 0 29 Sep 2024
Approximating Metric Magnitude of Point SetsAAAI Conference on Artificial Intelligence (AAAI), 2024 R. Andreeva James Ward Primoz Skraba Jie Gao Rik Sarkar 124 3 0 06 Sep 2024
Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesis Qi Sun Hang Zhou Wengang Zhou Li Li Houqiang Li 3DPC 3DV 265 12 0 07 Jul 2024
Paired Conditional Generative Adversarial Network for Highly Accelerated Liver 4D MRI Di Xu Xin Miao Hengjie Liu Jessica E. Scholey Wensha Yang ... Michael Ohliger Hui Lin Yi Lao Yang Yang Ke Sheng MedIm 171 5 0 20 May 2024
Neural Clustering based Visual Representation Learning Guikun Chen Xia Li Yi Yang Wenguan Wang SSL 310 14 0 26 Mar 2024
Do not trust what you trust: Miscalibration in Semi-supervised Learning Shambhavi Mishra Balamurali Murugesan Ismail Ben Ayed M. Pedersoli Jose Dolz 228 2 0 22 Mar 2024
Frozen Feature Augmentation for Few-Shot Image Classification Andreas Bär N. Houlsby Mostafa Dehghani Manoj Kumar VLM 285 15 0 15 Mar 2024
Convolutional Initialization for Data-Efficient Vision Transformers Jianqiao Zheng Xueqian Li Simon Lucey 260 2 0 23 Jan 2024
TULIP: Transformer for Upsampling of LiDAR Point Clouds Bin Yang Patrick Pfreundschuh Roland Siegwart Marco Hutter Peyman Moghadam Vaishakh Patil 3DPC 319 13 0 11 Dec 2023
Initializing Models with Larger OnesInternational Conference on Learning Representations (ICLR), 2023 Zhiqiu Xu Yanjie Chen Kirill Vishniakov Yida Yin Zhiqiang Shen Trevor Darrell Lingjie Liu Zhuang Liu 237 28 0 30 Nov 2023
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked AutoencodersIEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2023 Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 288 23 0 31 Oct 2023
Scalarization for Multi-Task and Multi-Domain Learning at ScaleNeural Information Processing Systems (NeurIPS), 2023 Amelie Royer Tijmen Blankevoort B. Bejnordi 243 25 0 13 Oct 2023
Mitigating the Effect of Incidental Correlations on Part-based LearningNeural Information Processing Systems (NeurIPS), 2023 Gaurav Bhatt Deepayan Das Leonid Sigal Vineeth N. Balasubramanian 206 7 0 30 Sep 2023
Asca: less audio data is more insightful Xiang Li Jing Chen Chao Li Hongwu Lv 113 0 0 23 Sep 2023
YOLOPose V2: Understanding and Improving Transformer-based 6D Pose Estimation Arul Selvam Periyasamy Arash A. Amini Vladimir Tsaturyan Sven Behnke ViT 186 19 0 21 Jul 2023
Learnable Weight Initialization for Volumetric Medical Image Segmentation Shahina Kunhimon Abdelrahman M. Shaker Muzammal Naseer Salman Khan Fahad Shahbaz Khan 201 1 0 15 Jun 2023
Mimetic Initialization of Self-Attention LayersInternational Conference on Machine Learning (ICML), 2023 Asher Trockman J. Zico Kolter 226 43 0 16 May 2023
Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models Qiufeng Wang Xu Yang Shuxia Lin Jing Wang Xin Geng 197 18 0 03 May 2023
Astroformer: More Data Might not be all you need for Classification Rishit Dagli 353 12 0 03 Apr 2023
Supervised Masked Knowledge Distillation for Few-Shot TransformersComputer Vision and Pattern Recognition (CVPR), 2023 Hanxi Lin G. Han Jiawei Ma Shiyuan Huang Xudong Lin Shih-Fu Chang 356 48 0 25 Mar 2023
Masked autoencoders are effective solution to transformer data-hungry Jia-ju Mao Honggu Zhou Xuesong Yin Binling Nie MedIm 411 9 0 12 Dec 2022
MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous AttentionIEEE International Conference on Computer Vision (ICCV), 2022 Wenyuan Zeng Meng Li Wenjie Xiong Tong Tong Wen-jie Lu Jin Tan Runsheng Wang Ru Huang 313 33 0 25 Nov 2022
Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets Xiangyu Chen Ying Qin Wenju Xu A. Bur Cuncong Zhong Guanghui Wang ViT 144 3 0 25 Oct 2022