Stand-Alone Self-Attention in Vision Models

13 June 2019

Papers citing "Stand-Alone Self-Attention in Vision Models"

34 / 234 papers shown

Title
Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth Yihe Dong Jean-Baptiste Cordonnier Andreas Loukas 32 373 0 05 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 53 973 0 04 Mar 2021
Generative Adversarial Transformers Drew A. Hudson C. L. Zitnick ViT 25 179 0 01 Mar 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 269 179 0 17 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,982 0 09 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Tsung-Yi Lin Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 979 0 27 Jan 2021
Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation Ye Huang Di Kang W. Jia Xiangjian He Liu Liu 31 35 0 19 Jan 2021
Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification Ardhendu Behera Zachary Wharton Pradeep Ruwan Padmasiri Galbokka Hewage Asish Bera 61 108 0 17 Jan 2021
Neural Sequence-to-grid Module for Learning Symbolic Rules Segwang Kim Hyoungwook Nam Joonyoung Kim Kyomin Jung NAI 64 11 0 13 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 227 2,428 0 04 Jan 2021
AttentionLite: Towards Efficient Self-Attention Models for Vision Souvik Kundu Sairam Sundaresan 16 22 0 21 Dec 2020
Convolutional LSTM Neural Networks for Modeling Wildland Fire Dynamics J. Burge M. Bonanni M. Ihme Lily Hu 19 19 0 11 Dec 2020
Deep Learning and the Global Workspace Theory R. V. Rullen Ryota Kanai 37 65 0 04 Dec 2020
Covariance Self-Attention Dual Path UNet for Rectal Tumor Segmentation Haijun Gao Bochuan Zheng Dazhi Pan Xiangyin Zeng 22 2 0 04 Nov 2020
Real-time Semantic Segmentation with Context Aggregation Network M. Yang Saumya Kumaar Ye Lyu F. Nex SSeg 35 63 0 02 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 41 39,266 0 22 Oct 2020
Bayesian Attention Modules Xinjie Fan Shujian Zhang Bo Chen Mingyuan Zhou 111 59 0 20 Oct 2020
Attention Augmented ConvLSTM for Environment Prediction Bernard Lange Masha Itkina Mykel J. Kochenderfer 76 21 0 19 Oct 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 82 1,101 0 14 Sep 2020
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks F. Fuchs Daniel E. Worrall Volker Fischer Max Welling 3DPC 45 665 0 18 Jun 2020
The Lipschitz Constant of Self-Attention Hyunjik Kim George Papamakarios A. Mnih 14 134 0 08 Jun 2020
Visual Transformers: Token-based Image Representation and Processing for Computer Vision Bichen Wu Chenfeng Xu Xiaoliang Dai Alvin Wan Peizhao Zhang Zhicheng Yan M. Tomizuka Joseph E. Gonzalez Kurt Keutzer Peter Vajda ViT 33 545 0 05 Jun 2020
Normalized Attention Without Probability Cage Oliver Richter Roger Wattenhofer 14 21 0 19 May 2020
Salient Object Detection Combining a Self-attention Module and a Feature Pyramid Network Guangyu Ren Tianhong Dai P. Barmpoutis Tania Stathaki 16 15 0 30 Apr 2020
Exploring Self-attention for Image Recognition Hengshuang Zhao Jiaya Jia V. Koltun SSL 26 772 0 28 Apr 2020
Experience Grounds Language Yonatan Bisk Ari Holtzman Jesse Thomason Jacob Andreas Yoshua Bengio ... Angeliki Lazaridou Jonathan May Aleksandr Nisnevich Nicolas Pinto Joseph P. Turian 19 350 0 21 Apr 2020
Hierarchical Opacity Propagation for Image Matting Yaoyi Li Qin Xu Hongtao Lu 22 13 0 07 Apr 2020
PIC: Permutation Invariant Convolution for Recognizing Long-range Activities Noureldien Hussein E. Gavves A. Smeulders VLM 18 13 0 18 Mar 2020
Electricity Theft Detection with self-attention Paulo Finardi Israel Campiotti Gustavo Plensack R. Souza Rodrigo Nogueira G. Pinheiro R. Lotufo 20 29 0 14 Feb 2020
Attentive Group Equivariant Convolutional Networks David W. Romero Erik J. Bekkers Jakub M. Tomczak Mark Hoogendoorn 22 89 0 07 Feb 2020
LIP: Local Importance-based Pooling Ziteng Gao Limin Wang Gangshan Wu FAtt 29 94 0 12 Aug 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 34 1,000 0 22 Apr 2019
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016