Efficient Attention: Attention with Linear Complexities

4 December 2018

Papers citing "Efficient Attention: Attention with Linear Complexities"

27 / 77 papers shown

Title
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation Qihang Yu Huiyu Wang Dahun Kim Siyuan Qiao Maxwell D. Collins Yukun Zhu Hartwig Adam Alan Yuille Liang-Chieh Chen ViT MedIm 32 89 0 17 Jun 2022
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 16 25 0 17 Jun 2022
Dynamic Linear Transformer for 3D Biomedical Image Segmentation Zheyu Zhang Ulas Bagci ViT MedIm 17 12 0 01 Jun 2022
Fair Comparison between Efficient Attentions Jiuk Hong Chaehyeon Lee Soyoun Bang Heechul Jung 17 1 0 01 Jun 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 36 149 0 27 Apr 2022
Efficient Linear Attention for Fast and Accurate Keypoint Matching Suwichaya Suwanwimolkul S. Komorita 3DPC 3DV 19 11 0 16 Apr 2022
A Call for Clarity in Beam Search: How It Works and When It Stops Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Dragomir R. Radev Yejin Choi Noah A. Smith 26 6 0 11 Apr 2022
MatchFormer: Interleaving Attention in Transformers for Feature Matching Qing Wang Jiaming Zhang Kailun Yang Kunyu Peng Rainer Stiefelhagen ViT 33 141 0 17 Mar 2022
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers Jiaming Zhang Huayao Liu Kailun Yang Xinxin Hu Ruiping Liu Rainer Stiefelhagen ViT 21 296 0 09 Mar 2022
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging Xiaowan Hu Yuanhao Cai Jing Lin Haoqian Wang X. Yuan Yulun Zhang Radu Timofte Luc Van Gool 29 134 0 04 Mar 2022
Self-attention Does Not Need $O(n^2)$ Memory M. Rabe Charles Staats LRM 18 139 0 10 Dec 2021
A Multi-attribute Controllable Generative Model for Histopathology Image Synthesis Jiarong Ye Yuan Xue Peter Liu R. Zaino K. Cheng Xiaolei Huang MedIm 22 7 0 10 Nov 2021
Classification of hierarchical text using geometric deep learning: the case of clinical trials corpus Sohrab Ferdowsi Nikolay Borissov J. Knafou P. Amini Douglas Teodoro 16 7 0 04 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 106 20 0 29 Sep 2021
Anchor DETR: Query Design for Transformer-Based Object Detection Yingming Wang X. Zhang Tong Yang Jian-jun Sun ViT 8 53 0 15 Sep 2021
Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximation Samuel Cahyawijaya 26 12 0 24 Aug 2021
CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation Y. Tashiro Jiaming Song Yang Song Stefano Ermon BDL DiffM 11 513 0 07 Jul 2021
Polarized Self-Attention: Towards High-quality Pixel-wise Regression Huajun Liu Fuqiang Liu Xinyi Fan Dong Huang 72 211 0 02 Jul 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 49 1,167 0 09 Jun 2021
Choose a Transformer: Fourier or Galerkin Shuhao Cao 34 220 0 31 May 2021
Relative Positional Encoding for Transformers with Linear Complexity Antoine Liutkus Ondřej Cífka Shih-Lun Wu Umut Simsekli Yi-Hsuan Yang Gaël Richard 25 44 0 18 May 2021
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols Songlin Yang Yanpeng Zhao Kewei Tu 18 22 0 28 Apr 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 38 2,163 0 20 Apr 2021
Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag Kazuki Irie Jürgen Schmidhuber 14 221 0 22 Feb 2021
Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images Rui Li Shunyi Zheng Chenxi Duan Jianlin Su Ce Zhang 17 187 0 29 Nov 2020
Sparsifying Transformer Models with Trainable Representation Pooling Michal Pietruszka Łukasz Borchmann Lukasz Garncarek 13 10 0 10 Sep 2020
Cross Attention Network for Few-shot Classification Rui Hou Hong Chang Bingpeng Ma Shiguang Shan Xilin Chen 202 629 0 17 Oct 2019