Augmented Shortcuts for Vision Transformers

30 June 2021

Papers citing "Augmented Shortcuts for Vision Transformers"

20 / 20 papers shown

Title
Learning Object Focused Attention Vivek Trivedy A. Almalki Longin Jan Latecki 31 0 0 10 Apr 2025
Generalized Probabilistic Attention Mechanism in Transformers DongNyeong Heo Heeyoul Choi 49 0 0 21 Oct 2024
Attacking Transformers with Feature Diversity Adversarial Perturbation Chenxing Gao Hang Zhou Junqing Yu Yuteng Ye Jiale Cai Junle Wang Wei Yang AAML 32 3 0 10 Mar 2024
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 20 5 0 09 Jan 2024
PanGu- $π$ : Enhancing Language Model Architectures via Nonlinearity Compensation Yunhe Wang Hanting Chen Yehui Tang Tianyu Guo Kai Han ... Qinghua Xu Qun Liu Jun Yao Chao Xu Dacheng Tao 59 15 0 27 Dec 2023
Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection Reza Azad A. Kazerouni Babak Azad Ehsan Khodapanah Aghdam Yury Velichko Ulas Bagci Dorit Merhof ViT MedIm 11 16 0 31 Aug 2023
GhostNetV2: Enhance Cheap Operation with Long-Range Attention Yehui Tang Kai Han Jianyuan Guo Chang Xu Chaoting Xu Yunhe Wang 11 270 0 23 Nov 2022
Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers Sifan Long Z. Zhao Jimin Pi Sheng-sheng Wang Jingdong Wang 11 29 0 21 Nov 2022
3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume Jianye Pang Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao ViT MedIm 26 11 0 14 Apr 2022
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice Peihao Wang Wenqing Zheng Tianlong Chen Zhangyang Wang ViT 9 127 0 09 Mar 2022
PyramidTNT: Improved Transformer-in-Transformer Baselines with Pyramid Architecture Kai Han Jianyuan Guo Yehui Tang Yunhe Wang ViT 24 22 0 04 Jan 2022
An Image Patch is a Wave: Phase-Aware Vision MLP Yehui Tang Kai Han Jianyuan Guo Chang Xu Yanxi Li Chao Xu Yunhe Wang 16 133 0 24 Nov 2021
Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation Jiaqi Gu Hyoukjun Kwon Dilin Wang Wei Ye Meng Li Yu-Hsin Chen Liangzhen Lai Vikas Chandra D. Pan ViT 11 180 0 01 Nov 2021
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 337 500 0 13 Jul 2021
Post-Training Quantization for Vision Transformer Zhenhua Liu Yunhe Wang Kai Han Siwei Ma Wen Gao ViT MQ 37 321 0 27 Jun 2021
Patch Slimming for Efficient Vision Transformers Yehui Tang Kai Han Yunhe Wang Chang Xu Jianyuan Guo Chao Xu Dacheng Tao ViT 16 163 0 05 Jun 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,523 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,604 0 24 Feb 2021
A Survey on Visual Transformer Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo ... Chunjing Xu Yixing Xu Zhaohui Yang Yiman Zhang Dacheng Tao ViT 16 2,123 0 23 Dec 2020
Distilling Knowledge from Graph Convolutional Networks Yiding Yang Jiayan Qiu Mingli Song Dacheng Tao Xinchao Wang 146 226 0 23 Mar 2020