DeepViT: Towards Deeper Vision Transformer

22 March 2021

Linjie Yang

Papers citing "DeepViT: Towards Deeper Vision Transformer"

50 / 253 papers shown

Title
Bootstrapping ViTs: Towards Liberating Vision Transformers from Pre-training Haofei Zhang Jiarui Duan Mengqi Xue Jie Song Li Sun Mingli Song ViT AI4CE 8 16 0 07 Dec 2021
Learning Tracking Representations via Dual-Branch Fully Transformer Networks Fei Xie Chunyu Wang Guangting Wang Wankou Yang Wenjun Zeng ViT 14 47 0 05 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 36 129 0 02 Dec 2021
SwinTrack: A Simple and Strong Baseline for Transformer Tracking Liting Lin Heng Fan Zhipeng Zhang Yong-mei Xu Haibin Ling ViT 23 301 0 02 Dec 2021
Adaptive Token Sampling For Efficient Vision Transformers Mohsen Fayyaz Soroush Abbasi Koohpayegani F. Jafari Sunando Sengupta Hamid Reza Vaezi Joze Eric Sommerlade Hamed Pirsiavash Juergen Gall ViT 16 146 0 30 Nov 2021
Shunted Self-Attention via Multi-Scale Token Aggregation Sucheng Ren Daquan Zhou Shengfeng He Jiashi Feng Xinchao Wang ViT 25 222 0 30 Nov 2021
Self-slimmed Vision Transformer Zhuofan Zong Kunchang Li Guanglu Song Yali Wang Yu Qiao B. Leng Yu Liu ViT 16 30 0 24 Nov 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 29 40 0 23 Nov 2021
INTERN: A New Learning Paradigm Towards General Vision Jing Shao Siyu Chen Yangguang Li Kun Wang Zhen-fei Yin ... F. Yu Junjie Yan Dahua Lin Xiaogang Wang Yu Qiao 8 34 0 16 Nov 2021
Attention Mechanisms in Computer Vision: A Survey Meng-Hao Guo Tianhan Xu Jiangjiang Liu Zheng-Ning Liu Peng-Tao Jiang Tai-Jiang Mu Song-Hai Zhang Ralph Robert Martin Ming-Ming Cheng Shimin Hu 11 1,626 0 15 Nov 2021
Transformer-based Image Compression Ming-Tse Lu Peiyao Guo Huiqing Shi Chuntong Cao Zhan Ma ViT 57 103 0 12 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 69 330 0 11 Nov 2021
Sliced Recursive Transformer Zhiqiang Shen Zechun Liu Eric P. Xing ViT 17 27 0 09 Nov 2021
The Nuts and Bolts of Adopting Transformer in GANs Rui Xu Xiangyu Xu Kai-xiang Chen Bolei Zhou Chen Change Loy ViT 12 4 0 25 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 28 38 0 10 Oct 2021
TranSalNet: Towards perceptually relevant visual saliency prediction Jianxun Lou Hanhe Lin David Marshall Dietmar Saupe Hantao Liu ViT 16 75 0 07 Oct 2021
Universality of Winning Tickets: A Renormalization Group Perspective William T. Redman Tianlong Chen Zhangyang Wang Akshunna S. Dogra UQCV 54 7 0 07 Oct 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 189 1,210 0 05 Oct 2021
Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? Chuanxin Tang Yucheng Zhao Guangting Wang Chong Luo Wenxuan Xie Wenjun Zeng MoE ViT 27 98 0 12 Sep 2021
Towards Transferable Adversarial Attacks on Vision Transformers Zhipeng Wei Jingjing Chen Micah Goldblum Zuxuan Wu Tom Goldstein Yu-Gang Jiang ViT AAML 19 111 0 09 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 13 41 0 08 Sep 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 172 475 0 12 Aug 2021
RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality? Yuki Tatsunami Masato Taki 19 12 0 09 Aug 2021
AS-MLP: An Axial Shifted MLP Architecture for Vision Dongze Lian Zehao Yu Xing Sun Shenghua Gao 12 189 0 18 Jul 2021
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation Zhiwei Hao Jianyuan Guo Ding Jia Kai Han Yehui Tang Chao Zhang Dacheng Tao Yunhe Wang ViT 33 67 0 03 Jul 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 39 428 0 01 Jul 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 41 313 0 24 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 24 205 0 23 Jun 2021
More than Encoder: Introducing Transformer Decoder to Upsample Yijiang Li Wentian Cai Ying Gao Chengming Li Xiping Hu ViT MedIm 27 50 0 20 Jun 2021
A Stronger Baseline for Ego-Centric Action Detection Zhiwu Qing Ziyuan Huang Xiang Wang Yutong Feng Shiwei Zhang Jianwen Jiang Mingqian Tang Changxin Gao M. Ang Nong Sang EgoV 28 3 0 13 Jun 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 27 149 0 10 Jun 2021
Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition Ziyuan Huang Zhiwu Qing Xiang Wang Yutong Feng Shiwei Zhang Jianwen Jiang Zhurong Xia Mingqian Tang Nong Sang M. Ang ViT 17 11 0 09 Jun 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 31 1,167 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 18 1,058 0 08 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 13 215 0 08 Jun 2021
On the Connection between Local Attention and Dynamic Depth-wise Convolution Qi Han Zejia Fan Qi Dai Lei-huan Sun Ming-Ming Cheng Jiaying Liu Jingdong Wang ViT 10 104 0 08 Jun 2021
Person Re-Identification with a Locally Aware Transformer Charu Sharma S. R. Kapil David Chapman ViT 24 45 0 07 Jun 2021
Refiner: Refining Self-attention for Vision Transformers Daquan Zhou Yujun Shi Bingyi Kang Weihao Yu Zihang Jiang Yuan Li Xiaojie Jin Qibin Hou Jiashi Feng ViT 12 59 0 07 Jun 2021
Vision Transformers with Hierarchical Attention Yun-Hai Liu Yu-Huan Wu Guolei Sun Le Zhang Ajad Chhatkuli Luc Van Gool ViT 22 32 0 06 Jun 2021
NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training Adam Yala H. Esfahanizadeh Rafael G. L. DÓliveira K. Duffy M. Ghobadi Tommi Jaakkola Vinod Vaikuntanathan Regina Barzilay Muriel Médard OOD FedML 24 22 0 04 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 18 664 0 03 Jun 2021
Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model Jiangning Zhang Chao Xu Jian Li Wenzhou Chen Yabiao Wang Ying Tai Shuo Chen Chengjie Wang Feiyue Huang Yong Liu 27 22 0 31 May 2021
SDNet: mutil-branch for single image deraining using swin Fuxiang Tan Yuting Kong Yingying Fan Feng Liu Daxin Zhou Hao Zhang Long Chen Liang Gao Yurong Qian ViT 23 15 0 31 May 2021
Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition Yulin Wang Rui Huang S. Song Zeyi Huang Gao Huang ViT 17 188 0 31 May 2021
Dual-stream Network for Visual Recognition Mingyuan Mao Renrui Zhang Honghui Zheng Peng Gao Teli Ma Yan Peng Errui Ding Baochang Zhang Shumin Han ViT 18 63 0 31 May 2021
Less is More: Pay Less Attention in Vision Transformers Zizheng Pan Bohan Zhuang Haoyu He Jing Liu Jianfei Cai ViT 19 82 0 29 May 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 32 105 0 28 May 2021
Vision Transformers with Patch Diversification Chengyue Gong Dilin Wang Meng Li Vikas Chandra Qiang Liu ViT 37 62 0 26 Apr 2021
All Tokens Matter: Token Labeling for Training Better Vision Transformers Zihang Jiang Qibin Hou Li-xin Yuan Daquan Zhou Yujun Shi Xiaojie Jin Anran Wang Jiashi Feng ViT 12 203 0 22 Apr 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Tsung-Yi Lin Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 270 979 0 27 Jan 2021