Refiner: Refining Self-attention for Vision Transformers

7 June 2021

Weihao Yu

Papers citing "Refiner: Refining Self-attention for Vision Transformers"

37 / 37 papers shown

Title
P-TAME: Explain Any Image Classifier with Trained Perturbations Mariano V. Ntrougkas Vasileios Mezaris Ioannis Patras AAML FAtt 101 0 0 29 Jan 2025
Foundation Models for Autonomous Robots in Unstructured Environments Hossein Naderi Alireza Shojaei Lifu Huang LM&Ro 143 4 0 19 Jul 2024
A Manifold Representation of the Key in Vision Transformers Li Meng Morten Goodwin Anis Yazidi P. Engelstad 157 0 0 01 Feb 2024
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 162 8 0 09 Jan 2024
Large language models in healthcare and medical domain: A review Zabir Al Nazi Wei Peng LM&MA 139 226 0 12 Dec 2023
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 202 8 0 07 Dec 2023
Toward a Deeper Understanding: RetNet Viewed through Convolution Chenghao Li Chaoning Zhang ViT 119 10 0 11 Sep 2023
ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation Xian Lin Zengqiang Yan Xianbo Deng Chuansheng Zheng Li Yu ViT MedIm 140 42 0 09 Sep 2023
SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation Guhnoo Yun J. Yoo Kijung Kim Jeongho Lee Dong Hwan Kim MoE 102 19 0 22 Aug 2023
FFT-based Dynamic Token Mixer for Vision Yuki Tatsunami Masato Taki 171 38 0 07 Mar 2023
EIT: Enhanced Interactive Transformer Tong Zheng Bei Li Huiwen Bao Tong Xiao Jingbo Zhu 186 2 0 20 Dec 2022
Adaptive Attention Link-based Regularization for Vision TransformersAdvanced Video and Signal Based Surveillance (AVSS), 2024 Heegon Jin Jongwon Choi ViT 138 0 0 25 Nov 2022
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Zhaoyang Han Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng Hao Sun Xue Lin Zhenman Fang Yanzhi Wang ViT 151 84 0 15 Nov 2022
MetaFormer Baselines for Vision Weihao Yu Chenyang Si Pan Zhou Mi Luo Yichen Zhou Jiashi Feng Shuicheng Yan Xinchao Wang MoE 164 207 0 24 Oct 2022
Sequencer: Deep LSTM for Image Classification Yuki Tatsunami Masato Taki VLM ViT 158 92 0 04 May 2022
Understanding The Robustness in Vision Transformers Daquan Zhou Zhiding Yu Enze Xie Chaowei Xiao Anima Anandkumar Jiashi Feng J. Álvarez ViT 204 211 0 26 Apr 2022
A New Dataset and Transformer for Stereoscopic Video Super-Resolution H. Imani Md Baharul Islam Lai-Kuan Wong ViT 169 11 0 21 Apr 2022
The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training Hao Liu Xinghua Jiang Xin Li Antai Guo Deqiang Jiang Bo Ren 120 41 0 18 Apr 2022
3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume Jianye Pang Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao ViT MedIm 79 11 0 14 Apr 2022
Focal Modulation Networks Jianwei Yang Chunyuan Li Xiyang Dai Lu Yuan Jianfeng Gao 3DPC 217 319 0 22 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 146 45 0 12 Mar 2022
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice Peihao Wang Wenqing Zheng Tianlong Chen Zhangyang Wang ViT 123 162 0 09 Mar 2022
Dynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attention Kai Liu Tianyi Wu Cong Liu Guodong Guo ViT 162 20 0 08 Mar 2022
DynaMixer: A Vision MLP Architecture with Dynamic Mixing Ziyu Wang Wenhao Jiang Yiming Zhu Li Yuan Yibing Song Wei Liu 120 46 0 28 Jan 2022
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention Sitong Wu Tianyi Wu Hao Hao Tan G. Guo ViT 134 76 0 28 Dec 2021
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Zhenglun Kong Zhaoyang Han Xiaolong Ma Xin Meng Mengshu Sun ... Geng Yuan Bin Ren Minghai Qin Hao Tang Yanzhi Wang ViT 135 171 0 27 Dec 2021
ELSA: Enhanced Local Self-Attention for Vision Transformer Jingkai Zhou Pichao Wang Fan Wang Qiong Liu Hao Li Rong Jin ViT 143 42 0 23 Dec 2021
MetaFormer Is Actually What You Need for Vision Weihao Yu Mi Luo Pan Zhou Chenyang Si Yichen Zhou Xinchao Wang Jiashi Feng Shuicheng Yan 321 1,017 0 22 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Peng Wang Jianping Fan Zhiqiang He 3DGS ViT 294 396 0 11 Nov 2021
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation Tongkun Xu Weihua Chen Pichao Wang Fan Wang Hao Li Rong Jin ViT 258 243 0 13 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Qingsong Wen Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li Rong Jin 140 48 0 08 Sep 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 453 532 0 12 Aug 2021
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer Yifan Xu Zhijie Zhang Mengdan Zhang Kekai Sheng Ke Li Weiming Dong Liqing Zhang Changsheng Xu Xing Sun ViT 169 232 0 03 Aug 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 276 347 0 24 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 176 219 0 23 Jun 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Qingsong Wen F. Wang Ming Lin Shuning Chang Hao Li Rong Jin ViT 181 114 0 28 May 2021
A Survey on Visual Transformer Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo ... Chunjing Xu Yixing Xu Zhaohui Yang Yiman Zhang Dacheng Tao ViT 560 2,573 0 23 Dec 2020