Training data-efficient image transformers & distillation through attention

23 December 2020

Alexandre Sablayrolles

Hervé Jégou

ViT

ArXiv PDF HTML

Papers citing "Training data-efficient image transformers & distillation through attention"

50 / 983 papers shown

Title
Knowledge-Informed Deep Learning for Irrigation Type Mapping from Remote Sensing Oishee Bintey Hoque Nibir Chandra Mandal Abhijin Adiga S. Swarup S. Nouwakpo Amanda Wilson M. Marathe 31 0 0 13 May 2025
Achieving 3D Attention via Triplet Squeeze and Excitation Block Maan Alhazmi Abdulrahman Altahhan 21 0 0 09 May 2025
Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet Kodai Hirata Tsuyoshi Okita ViT 41 0 0 09 May 2025
From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection Moritz Vandenhirtz Julia E. Vogt 29 0 0 09 May 2025
Pro2SAM: Mask Prompt to SAM with Grid Points for Weakly Supervised Object Localization Xi Yang Songsong Duan Nannan Wang Xinbo Gao WSOL 73 0 0 08 May 2025
SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer Young-Hu Park R.-H. Park Hyung-Min Park 49 0 0 07 May 2025
Balancing Accuracy, Calibration, and Efficiency in Active Learning with Vision Transformers Under Label Noise Moseli Motsóehli Hope Mogale Kyungim Baek 38 0 0 07 May 2025
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs Lukas Meiner Jens Mehnert A. P. Condurache MQ 39 0 0 06 May 2025
Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction Changjun Li Runqing Jiang Zhuo Song Pengpeng Yu Ye Zhang Yulan Guo MQ 49 0 0 01 May 2025
GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretability Sehyeong Jo Gangjae Jang Haesol Park 32 0 0 28 Apr 2025
Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems Alireza Furutanpey Carmen Walser Philipp Raith P. Frangoudis Schahram Dustdar GNN NAI 83 0 0 28 Apr 2025
Enhancing Surgical Documentation through Multimodal Visual-Temporal Transformers and Generative AI Hugo Georgenthum Cristian Cosentino Fabrizio Marozzo Pietro Liò MedIm 114 0 0 28 Apr 2025
MERA: Multimodal and Multiscale Self-Explanatory Model with Considerably Reduced Annotation for Lung Nodule Diagnosis Jiahao Lu Chong Yin Silvia Ingala Kenny Erleben M. Nielsen S. Darkner 49 0 0 27 Apr 2025
WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada Braeden Sherritt Isar Nejadgholi Marzieh Amini VLM 44 0 0 17 Apr 2025
How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions Aditya Prakash Benjamin Lundell Dmitry Andreychuk David Forsyth Saurabh Gupta H. Sawhney 31 0 0 16 Apr 2025
Search is All You Need for Few-shot Anomaly Detection Qishan Wang Jia Guo Shuyong Gao H. Wang Li Xiong J. Hu Hanqi Guo Wenqiang Zhang 53 0 0 16 Apr 2025
Evolved Hierarchical Masking for Self-Supervised Learning Zhanzhou Feng Shiliang Zhang 37 0 0 12 Apr 2025
Low-Bit Integerization of Vision Transformers using Operand Reodering for Efficient Hardware Ching-Yi Lin Sahil Shah MQ 64 0 0 11 Apr 2025
Heart Failure Prediction using Modal Decomposition and Masked Autoencoders for Scarce Echocardiography Databases Andrés Bell-Navas M. Villalba-Orero Enrique Lara Pezzi J. Garicano-Mena S. L. Clainche 51 0 0 10 Apr 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 48 0 0 03 Apr 2025
HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning Hao Wang Shuo Zhang Biao Leng ViT 67 0 0 03 Apr 2025
Mamba-3D as Masked Autoencoders for Accurate and Data-Efficient Analysis of Medical Ultrasound Videos Jiaheng Zhou Yanfeng Zhou Wei Fang Yuxing Tang Le Lu Ge Yang Mamba 182 0 0 26 Mar 2025
Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding Jinlong Li Cristiano Saltori Fabio Poiesi N. Sebe 127 0 0 20 Mar 2025
Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimization Y. Zhang Yingzhe Xu Junyu Shi L. Zhang Shengshan Hu Minghui Li Yanjun Zhang AAML 45 1 0 17 Mar 2025
8-Calves Image dataset Xuyang Fang S. Hannuna Neill D. F. Campbell 104 0 0 17 Mar 2025
Self-Supervised Pretraining for Fine-Grained Plankton Recognition Joona Kareinen T. Eerola K. Kraft L. Lensu S. Suikkanen H. Kalviainen SSL 131 0 0 14 Mar 2025
ReSi: A Comprehensive Benchmark for Representational Similarity Measures Max Klabunde Tassilo Wald Tobias Schumacher Klaus H. Maier-Hein Markus Strohmaier Adriana Iamnitchi AI4TS VLM 68 5 0 13 Mar 2025
Accelerate 3D Object Detection Models via Zero-Shot Attention Key Pruning Lizhen Xu Xiuxiu Bai Xiaojun Jia Jianwu Fang Shanmin Pang 61 0 0 13 Mar 2025
Breaking the Limits of Quantization-Aware Defenses: QADT-R for Robustness Against Patch-Based Adversarial Attacks in QNNs Amira Guesmi B. Ouni Muhammad Shafique MQ AAML 36 0 0 10 Mar 2025
Spectral State Space Model for Rotation-Invariant Visual Representation Learning Sahar Dastani Ali Bahri Moslem Yazdanpanah Mehrdad Noori David Osowiechi ... Farzad Beizaee Milad Cheraghalikhani A. Mondal H. Lombaert Christian Desrosiers 50 0 0 09 Mar 2025
Nexar Dashcam Collision Prediction Dataset and Challenge Daniel C. Moura Shizhan Zhu Orly Zvitia 66 0 0 05 Mar 2025
Transformer Meets Twicing: Harnessing Unattended Residual Information Laziz U. Abdullaev Tan M. Nguyen 39 2 0 02 Mar 2025
Split Adaptation for Pre-trained Vision Transformers Lixu Wang Bingqi Shang Y. Li Payal Mohapatra Wei Dong Xiao-Xu Wang Qi Zhu ViT 43 0 0 01 Mar 2025
Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer Yufei Guo Xiaode Liu Y. Chen Weihang Peng Yuhan Zhang Zhe Ma MQ 43 0 0 28 Feb 2025
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 71 0 0 28 Feb 2025
OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels Meng Lou Yizhou Yu 110 1 0 27 Feb 2025
Examining the Threat Landscape: Foundation Models and Model Stealing Ankita Raj Deepankar Varma Chetan Arora AAML 68 1 0 25 Feb 2025
Vision-LSTM: xLSTM as Generic Vision Backbone Benedikt Alkin M. Beck Korbinian Poppel Sepp Hochreiter Johannes Brandstetter VLM 56 42 0 24 Feb 2025
Disentangling Visual Transformers: Patch-level Interpretability for Image Classification Guillaume Jeanneret Loïc Simon F. Jurie ViT 44 0 0 24 Feb 2025
Graph Perceiver IO: A General Architecture for Graph Structured Data Seyun Bae Hoyoon Byun Changdae Oh Yoon-Sik Cho Kyungwoo Song GNN 90 2 0 24 Feb 2025
Simplifying DINO via Coding Rate Regularization Ziyang Wu Jingyuan Zhang Druv Pai X. Wang Chandan Singh Jianwei Yang Jianfeng Gao Yi-An Ma 143 1 0 17 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 52 0 0 17 Feb 2025
ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Models Ehsan Zeraatkar Salah Faroughi Jelena Tešić 109 0 0 10 Feb 2025
Exploring Visual Embedding Spaces Induced by Vision Transformers for Online Auto Parts Marketplaces Cameron Armijo Pablo Rivas 39 0 0 09 Feb 2025
Kolmogorov-Arnold Fourier Networks Jusheng Zhang Yijia Fan Kaitong Cai Keze Wang 61 0 0 09 Feb 2025
All-in-One Image Compression and Restoration Huimin Zeng Jiacheng Li Ziqiang Zheng Zhiwei Xiong 83 1 0 05 Feb 2025
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation Tao Zhang Jinyong Wen Zhen Chen Kun Ding S. Xiang Chunhong Pan 72 1 0 04 Feb 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application Chuanyang Zheng ViT 67 0 0 26 Jan 2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers Weikang Meng Yadan Luo Xin Li D. Jiang Zheng Zhang 115 0 0 25 Jan 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles X. Ling Boyu Wang 47 1 0 24 Jan 2025