DaViT: Dual Attention Vision Transformers

7 April 2022

Mingyu Ding

Bin Xiao

Noel Codella

Ping Luo

Jingdong Wang

Lu Yuan

ViT

ArXiv PDF HTML

Papers citing "DaViT: Dual Attention Vision Transformers"

35 / 35 papers shown

Title
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 87 1 0 10 Mar 2025
CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution Xin Liu Jie Liu J. Tang Gangshan Wu SupR ViT 54 0 0 10 Mar 2025
VMamba: Visual State Space Model Yue Liu Yunjie Tian Yuzhong Zhao Hongtian Yu Lingxi Xie Yaowei Wang Qixiang Ye Jianbin Jiao Yunfan Liu Mamba 106 592 0 31 Dec 2024
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 28 0 0 12 Nov 2024
AceParse: A Comprehensive Dataset with Diverse Structured Texts for Academic Literature Parsing Huawei Ji Cheng Deng Bo Xue Zhouyang Jin Jiaxin Ding Xiaoying Gan Luoyi Fu Xinbing Wang Chenghu Zhou 20 0 0 16 Sep 2024
HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution Xiang Zhang Yulun Zhang Fisher Yu 29 15 0 08 Jul 2024
Vision Transformer with Sparse Scan Prior Qihang Fan Huaibo Huang Mingrui Chen Ran He ViT 36 4 0 22 May 2024
Tiny Models are the Computational Saver for Large Models Qingyuan Wang B. Cardiff Antoine Frappé Benoît Larras Deepu John 24 2 0 26 Mar 2024
TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classification Kaipeng Zheng Weiran Huang Lichao Sun LM&MA MedIm VLM 22 0 0 12 Dec 2023
SCHEME: Scalable Channel Mixer for Vision Transformers Deepak Sridhar Yunsheng Li Nuno Vasconcelos 18 0 0 01 Dec 2023
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Bin Xiao Haiping Wu Weijian Xu Xiyang Dai Houdong Hu Yumao Lu Michael Zeng Ce Liu Lu Yuan VLM 31 140 0 10 Nov 2023
Low-Resolution Self-Attention for Semantic Segmentation Yu-Huan Wu Shi-Chen Zhang Yun-Hai Liu Le Zhang Xin Zhan Daquan Zhou Jiashi Feng Ming-Ming Cheng Liangli Zhen ViT 32 3 0 08 Oct 2023
Dual Aggregation Transformer for Image Super-Resolution Zheng Chen Yulun Zhang Jinjin Gu L. Kong Xiaokang Yang F. I. F. Richard Yu ViT 11 163 0 07 Aug 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 18 116 0 25 Jul 2023
Lightweight Vision Transformer with Bidirectional Interaction Qihang Fan Huaibo Huang Xiaoqiang Zhou Ran He ViT 27 27 0 01 Jun 2023
Towards an Effective and Efficient Transformer for Rain-by-snow Weather Removal Tao Gao Yuanbo Wen Kaihao Zhang Peng Cheng Ting Chen ViT 21 5 0 06 Apr 2023
Vision Transformer with Quadrangle Attention Qiming Zhang Jing Zhang Yufei Xu Dacheng Tao ViT 19 38 0 27 Mar 2023
Transformadores: Fundamentos teoricos y Aplicaciones J. D. L. Torre 60 0 0 18 Feb 2023
Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing Nataniel Ruiz Sarah Adel Bargal Cihang Xie Kate Saenko Stan Sclaroff ViT 13 5 0 29 Nov 2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces Eric N. D. Nguyen Karan Goel Albert Gu Gordon W. Downs Preey Shah Tri Dao S. Baccus Christopher Ré VLM 22 37 0 12 Oct 2022
Transformer Vs. MLP-Mixer: Exponential Expressive Gap For NLP Problems D. Navon A. Bronstein MoE 20 0 0 17 Aug 2022
Focal Modulation Networks Jianwei Yang Chunyuan Li Xiyang Dai Lu Yuan Jianfeng Gao 3DPC 22 261 0 22 Mar 2022
Patches Are All You Need? Asher Trockman J. Zico Kolter ViT 214 395 0 24 Jan 2022
UniFormer: Unifying Convolution and Self-attention for Visual Recognition Kunchang Li Yali Wang Junhao Zhang Peng Gao Guanglu Song Yu Liu Hongsheng Li Yu Qiao ViT 142 360 0 24 Jan 2022
QuadTree Attention for Vision Transformers Shitao Tang Jiahui Zhang Siyu Zhu Ping Tan ViT 157 154 0 08 Jan 2022
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 106 20 0 29 Sep 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,554 0 04 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 109 206 0 26 Apr 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 260 178 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 220 510 0 11 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Tsung-Yi Lin Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 267 955 0 27 Jan 2021
Deep High-Resolution Representation Learning for Visual Recognition Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng ... Yadong Mu Mingkui Tan Xinggang Wang Wenyu Liu Bin Xiao 190 3,480 0 20 Aug 2019
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 261 10,106 0 16 Nov 2016