Early Convolutions Help Transformers See Better

Early Convolutions Help Transformers See Better

28 June 2021

Piotr Dollár

Ross B. Girshick

Papers citing "Early Convolutions Help Transformers See Better"

14 / 114 papers shown

Title
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 194 1,212 0 05 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 108 20 0 29 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 13 41 0 08 Sep 2021
Deep DNA Storage: Scalable and Robust DNA Storage via Coding Theory and Deep Learning Daniella Bar-Lev Itai Orr Omer Sabary T. Etzion Eitan Yaakobi 29 24 0 31 Aug 2021
Hire-MLP: Vision MLP via Hierarchical Rearrangement Jianyuan Guo Yehui Tang Kai Han Xinghao Chen Han Wu Chao Xu Chang Xu Yunhe Wang 40 105 0 30 Aug 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 51 779 0 24 Aug 2021
SwinIR: Image Restoration Using Swin Transformer Jingyun Liang Jie Cao Guolei Sun K. Zhang Luc Van Gool Radu Timofte ViT 42 2,806 0 23 Aug 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 177 476 0 12 Aug 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 32 105 0 28 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 111 209 0 26 Apr 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 176 686 0 22 Apr 2021
Bridging Global Context Interactions for High-Fidelity Image Completion Chuanxia Zheng Tat-Jen Cham Jianfei Cai Dinh Q. Phung ViT 35 78 0 02 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 277 3,622 0 24 Feb 2021
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,216 0 16 Nov 2016