Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks

5 May 2021

Papers citing "Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks"

26 / 26 papers shown

Title
From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs Jiliang Ni Jiachen Pu Zhongyi Yang Kun Zhou Hui Wang Xiaoliang Xiao Dakui Wang Xin Li Jingfeng Luo Conggang Hu 32 0 0 18 Apr 2025
Quantum Complex-Valued Self-Attention Model Fu Chen Qinglin Zhao Li Feng Longfei Tang Yangbin Lin Haitao Huang MQ 47 0 0 24 Mar 2025
Threshold Attention Network for Semantic Segmentation of Remote Sensing Images Wei Long Yongjun Zhang Zhongwei Cui Yujie Xu Xuexue Zhang 72 11 0 14 Jan 2025
SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge Hao Ding Tuxun Lu Yuqian Zhang Ruixing Liang Hongchao Shu ... Bo Wang Marcos Fernández-Rodríguez Estevao Lima João L. Vilaça Mathias Unberath 55 4 0 16 Jul 2024
Attributes Grouping and Mining Hashing for Fine-Grained Image Retrieval Xin Lu Shikun Chen Yichao Cao Xin Zhou Xiaobo Lu 16 12 0 10 Nov 2023
Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels Ching-Hsun Tseng S. Chien Po-Shen Wang Shin-Jye Lee Wei-Huan Hu Bin Pu Xiaojun Zeng 19 1 0 15 Aug 2023
Detection of Pavement Cracks by Deep Learning Models of Transformer and UNet Yu Zhang Lin Zhang UQCV MedIm ViT 16 8 0 25 Apr 2023
Towards Lightweight Cross-domain Sequential Recommendation via External Attention-enhanced Graph Convolution Network Jinyu Zhang Huichuan Duan Lei Guo Liancheng Xu Xinhua Wang GNN 12 4 0 07 Feb 2023
Long Range Pooling for 3D Large-Scale Scene Understanding Xiang-Li Li Meng-Hao Guo Tai-Jiang Mu Ralph Robert Martin Shiyong Hu 3DV 3DPC 11 2 0 17 Jan 2023
MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation Jiacheng Ruan Suncheng Xiang Mingye Xie Ting Liu Yuzhuo Fu 11 130 0 03 Nov 2022
Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification Junfei Xiao Yutong Bai Alan Yuille Zongwei Zhou MedIm ViT 30 59 0 23 Oct 2022
Valuing Vicinity: Memory attention framework for context-based semantic segmentation in histopathology Oliver Ester Fabian Horst C. Seibold J. Keyl Saskia Ting ... P. Ivanyi Viktor Grünwald J. Bräsen Jan Egger Jens Kleesiek 8 7 0 21 Oct 2022
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 14 24 0 17 Jun 2022
Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance Justin Sanchez Christopher Neff Hamed Tabkhi GNN 30 9 0 15 Jan 2022
Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? Chuanxin Tang Yucheng Zhao Guangting Wang Chong Luo Wenxuan Xie Wenjun Zeng MoE ViT 19 96 0 12 Sep 2021
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 54 134 0 09 Sep 2021
Rethinking Token-Mixing MLP for MLP-based Vision Backbone Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 32 26 0 28 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 13 204 0 23 Jun 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,518 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,604 0 24 Feb 2021
TransReID: Transformer-based Object Re-Identification Shuting He Haowen Luo Pichao Wang F. Wang Hao Li Wei Jiang ViT 213 788 0 08 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 225 2,404 0 04 Jan 2021
Spatial Pyramid Based Graph Reasoning for Semantic Segmentation Xia Li Yibo Yang Qijie Zhao Tiancheng Shen Zhouchen Lin Hong Liu SSeg GNN 102 161 0 23 Mar 2020
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation C. Qi Hao Su Kaichun Mo Leonidas J. Guibas 3DH 3DPC 3DV PINN 219 13,886 0 02 Dec 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 261 10,196 0 16 Nov 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 249 1,821 0 18 Aug 2016