ELSA: Enhanced Local Self-Attention for Vision Transformer

ELSA: Enhanced Local Self-Attention for Vision Transformer

23 December 2021

Jingkai Zhou

Fan Wang

Hao Li

Papers citing "ELSA: Enhanced Local Self-Attention for Vision Transformer"

19 / 19 papers shown

Title
Frequency Disentangled Features in Neural Image Compression Ali Zafari Atefeh Khoshkhahtinat P. Mehta Mohammad Saeed Ebrahimi Saadabadi Mohammad Akyash Nasser M. Nasrabadi 27 14 0 04 Aug 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 11 8 0 03 Feb 2023
Boosting vision transformers for image retrieval Chull Hwan Song Jooyoung Yoon Shunghyun Choi Yannis Avrithis ViT 17 31 0 21 Oct 2022
Effective Vision Transformer Training: A Data-Centric Perspective Benjia Zhou Pichao Wang Jun Wan Yan-Ni Liang Fan Wang 19 5 0 29 Sep 2022
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications Muhammad Maaz Abdelrahman M. Shaker Hisham Cholakkal Salman Khan Syed Waqas Zamir Rao Muhammad Anwer F. Khan ViT 25 182 0 21 Jun 2022
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 189 1,148 0 05 Oct 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 11 41 0 08 Sep 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 172 462 0 12 Aug 2021
PSViT: Better Vision Transformer via Token Pooling and Attention Sharing Boyu Chen Peixia Li Baopu Li Chuming Li Lei Bai Chen Lin Ming-hui Sun Junjie Yan Wanli Ouyang ViT 63 33 0 07 Aug 2021
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 328 500 0 13 Jul 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 32 105 0 28 May 2021
Decoupled Dynamic Filter Networks Jingkai Zhou Varun Jampani Zhixiong Pi Qiong Liu Ming-Hsuan Yang 43 107 0 29 Apr 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 109 206 0 26 Apr 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
TransReID: Transformer-based Object Re-Identification Shuting He Haowen Luo Pichao Wang F. Wang Hao Li Wei Jiang ViT 213 769 0 08 Feb 2021
TransTrack: Multiple Object Tracking with Transformer Pei Sun Jinkun Cao Yi-Xin Jiang Rufeng Zhang Enze Xie Zehuan Yuan Changhu Wang Ping Luo ViT VOT 241 555 0 31 Dec 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,214 0 17 Apr 2017
A Non-convex One-Pass Framework for Generalized Factorization Machine and Rank-One Matrix Sensing Ming Lin Jieping Ye 29 21 0 21 Aug 2016