On the Connection between Local Attention and Dynamic Depth-wise Convolution

8 June 2021

Ming-Ming Cheng

Jingdong Wang

Papers citing "On the Connection between Local Attention and Dynamic Depth-wise Convolution"

26 / 26 papers shown

Title
Adaptive Dual-domain Learning for Underwater Image Enhancement Lingtao Peng Liheng Bian 22 0 0 27 Apr 2025
VMamba: Visual State Space Model Yue Liu Yunjie Tian Yuzhong Zhao Hongtian Yu Lingxi Xie Yaowei Wang Qixiang Ye Jianbin Jiao Yunfan Liu Mamba 113 609 0 31 Dec 2024
Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers Shwai He Tao Ge Guoheng Sun Bowei Tian Xiaoyang Wang Ang Li MoE 46 1 0 17 Oct 2024
MALT: Multi-scale Action Learning Transformer for Online Action Detection Zhipeng Yang Ruoyu Wang Yang Tan Liping Xie OffRL 38 1 0 31 May 2024
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights Moein Heidari Reza Azad Sina Ghorbani Kolahi René Arimond Leon Niggemeier ... Afshin Bozorgpour Ehsan Khodapanah Aghdam A. Kazerouni I. Hacihaliloglu Dorit Merhof 41 7 0 28 Mar 2024
HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs Ting Yao Yehao Li Yingwei Pan Tao Mei ViT 23 15 0 18 Mar 2024
TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition Meng Lou Hong-Yu Zhou Sibei Yang Yizhou Yu Chuan Wu Yizhou Yu ViT 36 36 0 30 Oct 2023
Dual Aggregation Transformer for Image Super-Resolution Zheng Chen Yulun Zhang Jinjin Gu L. Kong Xiaokang Yang F. I. F. Richard Yu ViT 11 167 0 07 Aug 2023
Frequency Disentangled Features in Neural Image Compression Ali Zafari Atefeh Khoshkhahtinat P. Mehta Mohammad Saeed Ebrahimi Saadabadi Mohammad Akyash Nasser M. Nasrabadi 40 14 0 04 Aug 2023
InceptionNeXt: When Inception Meets ConvNeXt Weihao Yu Pan Zhou Shuicheng Yan Xinchao Wang 45 117 0 29 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 40 47 0 21 Mar 2023
DLGSANet: Lightweight Dynamic Local and Global Self-Attention Networks for Image Super-Resolution Xiang Li Jin-shan Pan Jinhui Tang Jiangxin Dong 37 31 0 05 Jan 2023
A Close Look at Spatial Modeling: From Attention to Convolution Xu Ma Huan Wang Can Qin Kunpeng Li Xing Zhao Jie Fu Yun Fu ViT 3DPC 17 11 0 23 Dec 2022
Reversible Column Networks Yuxuan Cai Yi Zhou Qi Han Jianjian Sun Xiangwen Kong Jun Yu Li Xiangyu Zhang VLM 29 53 0 22 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 25 159 0 15 Dec 2022
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition Qibin Hou Cheng Lu Mingg-Ming Cheng Jiashi Feng ViT 23 129 0 22 Nov 2022
MetaFormer Baselines for Vision Weihao Yu Chenyang Si Pan Zhou Mi Luo Yichen Zhou Jiashi Feng Shuicheng Yan Xinchao Wang MoE 23 156 0 24 Oct 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 16 346 0 02 Jun 2022
Unified GCNs: Towards Connecting GCNs with CNNs Ziyan Zhang Bo Jiang Bin Luo GNN 23 1 0 26 Apr 2022
3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume Jianye Pang Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao ViT MedIm 28 11 0 14 Apr 2022
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 244 2,600 0 04 May 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,523 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,622 0 24 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 227 2,428 0 04 Jan 2021
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,549 0 17 Apr 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 288 10,214 0 16 Nov 2016