CvT: Introducing Convolutions to Vision Transformers

29 March 2021

Lu Yuan

Lei Zhang

ViT

ArXiv PDF HTML

Papers citing "CvT: Introducing Convolutions to Vision Transformers"

50 / 287 papers shown

Title
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang C. Li Biao Wang Xihan Wei Lei Zhang M. Keuper Xia Hua ViT 29 15 0 15 Jun 2022
Efficient Adaptive Ensembling for Image Classification A. Bruno Davide Moroni M. Martinelli 18 18 0 15 Jun 2022
MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing Zhaofan Qiu Ting Yao Chong-Wah Ngo Tao Mei ViT 22 15 0 13 Jun 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 16 346 0 02 Jun 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 52 26 0 30 May 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 110 17 0 30 May 2022
WaveMix: A Resource-efficient Neural Network for Image Analysis Pranav Jeevan Kavitha Viswanathan S. AnanduA A. Sethi 13 20 0 28 May 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 22 68 0 26 May 2022
Concurrent Neural Tree and Data Preprocessing AutoML for Image Classification Anish Thite Mohan Dodda Pulak Agarwal Jason Zutty 23 3 0 25 May 2022
Inception Transformer Chenyang Si Weihao Yu Pan Zhou Yichen Zhou Xinchao Wang Shuicheng Yan ViT 26 187 0 25 May 2022
Boosting Camouflaged Object Detection with Dual-Task Interactive Transformer Zheng Liu Zhili Zhang Wei Yu Wu 25 46 0 21 May 2022
POViT: Vision Transformer for Multi-objective Design and Characterization of Nanophotonic Devices Xinyu Chen Renjie Li Yueyao Yu Yuanwen Shen Wenye Li Zhaoyu Zhang Yin Zhang ViT 20 1 0 17 May 2022
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization Luke Melas-Kyriazi Christian Rupprecht Iro Laina Andrea Vedaldi 28 159 0 16 May 2022
Activating More Pixels in Image Super-Resolution Transformer Xiangyu Chen Xintao Wang Jiantao Zhou Yu Qiao Chao Dong ViT 59 600 0 09 May 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao Teli Ma Hongsheng Li Ziyi Lin Jifeng Dai Yu Qiao ViT 19 121 0 08 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 25 180 0 06 May 2022
Symmetric Transformer-based Network for Unsupervised Image Registration Mingrui Ma Lei Song Yuanbo Xu Gui-Xian Liu ViT MedIm 11 36 0 28 Apr 2022
DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers Xianing Chen Qiong Cao Yujie Zhong Jing Zhang Shenghua Gao Dacheng Tao ViT 29 76 0 27 Apr 2022
Deeper Insights into the Robustness of ViTs towards Common Corruptions Rui Tian Zuxuan Wu Qi Dai Han Hu Yu-Gang Jiang ViT AAML 16 4 0 26 Apr 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 21 123 0 14 Apr 2022
Neighborhood Attention Transformer Ali Hassani Steven Walton Jiacheng Li Shengjia Li Humphrey Shi ViT AI4TS 34 251 0 14 Apr 2022
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation Wenqiang Zhang Zilong Huang Guozhong Luo Tao Chen Xinggang Wang Wenyu Liu Gang Yu Chunhua Shen ViT 19 198 0 12 Apr 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 30 240 0 07 Apr 2022
MixFormer: Mixing Features across Windows and Dimensions Qiang Chen Qiman Wu Jian Wang Qinghao Hu T. Hu Errui Ding Jian Cheng Jingdong Wang MDE ViT 16 101 0 06 Apr 2022
SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space Evangelos Chatzipantazis Stefanos Pertigkiozoglou Edgar Dobriban Kostas Daniilidis 3DPC 32 30 0 05 Apr 2022
CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection Yanan Zhang Jiaxin Chen Di Huang ViT 3DPC 29 59 0 01 Apr 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 39 0 29 Mar 2022
Transformers Meet Visual Learning Understanding: A Comprehensive Review Yuting Yang Licheng Jiao Xuantong Liu F. Liu Shuyuan Yang Zhixi Feng Xu Tang ViT MedIm 24 28 0 24 Mar 2022
Focal Modulation Networks Jianwei Yang Chunyuan Li Xiyang Dai Lu Yuan Jianfeng Gao 3DPC 22 263 0 22 Mar 2022
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer Rui Yang Hailong Ma Jie Wu Yansong Tang Xuefeng Xiao Min Zheng Xiu Li ViT 19 53 0 21 Mar 2022
HIPA: Hierarchical Patch Transformer for Single Image Super Resolution Qing Cai Yiming Qian Jinxing Li Junjie Lv Yee-Hong Yang Feng Wu Dafan Zhang 17 28 0 19 Mar 2022
Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning Yang He Weihan Liang Dongyang Zhao Hong-Yu Zhou Weifeng Ge Yizhou Yu Wenqiang Zhang ViT 25 45 0 17 Mar 2022
Deep Transformers Thirst for Comprehensive-Frequency Data R. Xia Chao Xue Boyu Deng Fang Wang Jingchao Wang ViT 25 0 0 14 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding X. Zhang Yi Zhou Jungong Han Guiguang Ding Jian-jun Sun VLM 47 528 0 13 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 33 37 0 12 Mar 2022
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice Peihao Wang Wenqing Zheng Tianlong Chen Zhangyang Wang ViT 22 127 0 09 Mar 2022
Dynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attention Kai Liu Tianyi Wu Cong Liu Guodong Guo ViT 33 17 0 08 Mar 2022
Stepwise Feature Fusion: Local Guides Global Jinfeng Wang Qiming Huang Feilong Tang Jia Meng Jionglong Su Sifan Song ViT MedIm 19 179 0 07 Mar 2022
3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification Dening Lu Qian Xie Linlin Xu Jonathan Li 3DV 16 67 0 02 Mar 2022
A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark Yunhe Gao Mu Zhou Ding Liu Zhennan Yan Shaoting Zhang Dimitris N. Metaxas ViT MedIm 18 68 0 28 Feb 2022
CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising Dayang Wang Fenglei Fan Zhan Wu R. Liu Fei-Yue Wang Hengyong Yu ViT MedIm 17 121 0 28 Feb 2022
Factorizer: A Scalable Interpretable Approach to Context Modeling for Medical Image Segmentation Pooya Ashtari Diana Sima L. De Lathauwer D. Sappey-Marinier F. Maes Sabine Van Huffel ViT MedIm 21 35 0 24 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz X. Wang ViT VLM 189 499 0 22 Feb 2022
Visual Attention Network Meng-Hao Guo Chengrou Lu Zheng-Ning Liu Ming-Ming Cheng Shiyong Hu ViT VLM 19 637 0 20 Feb 2022
Discriminability-enforcing loss to improve representation learning Florinel-Alin Croitoru Diana-Nicoleta Grigore Radu Tudor Ionescu FaML 14 1 0 14 Feb 2022
CATs++: Boosting Cost Aggregation with Convolutions and Transformers Seokju Cho Sunghwan Hong Seung Wook Kim ViT 19 34 0 14 Feb 2022
Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs Huangjie Zheng Pengcheng He Weizhu Chen Mingyuan Zhou 22 14 0 14 Feb 2022
Feature-level augmentation to improve robustness of deep neural networks to affine transformations A. Sandru Mariana-Iuliana Georgescu Radu Tudor Ionescu OOD 11 3 0 10 Feb 2022
LwPosr: Lightweight Efficient Fine-Grained Head Pose Estimation Naina Dhingra 16 16 0 07 Feb 2022
Towards an Analytical Definition of Sufficient Data Adam Byerly T. Kalganova 17 4 0 07 Feb 2022