Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

28 January 2021

Weihao Yu

Papers citing "Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet"

50 / 357 papers shown

Title
Shunted Self-Attention via Multi-Scale Token Aggregation Sucheng Ren Daquan Zhou Shengfeng He Jiashi Feng Xinchao Wang ViT 25 222 0 30 Nov 2021
On the Integration of Self-Attention and Convolution Xuran Pan Chunjiang Ge Rui Lu S. Song Guanfu Chen Zeyi Huang Gao Huang SSL 38 287 0 29 Nov 2021
SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya Michael S. Ryoo 25 6 0 26 Nov 2021
A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation Himashi Peiris Munawar Hayat Zhaolin Chen Gary Egan Mehrtash Harandi ViT MedIm 14 123 0 26 Nov 2021
Self-slimmed Vision Transformer Zhuofan Zong Kunchang Li Guanglu Song Yali Wang Yu Qiao B. Leng Yu Liu ViT 21 30 0 24 Nov 2021
An Image Patch is a Wave: Phase-Aware Vision MLP Yehui Tang Kai Han Jianyuan Guo Chang Xu Yanxi Li Chao Xu Yunhe Wang 24 133 0 24 Nov 2021
PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer Zitong Yu Yuming Shen Jingang Shi Hengshuang Zhao Philip H. S. Torr Guoying Zhao ViT MedIm 134 167 0 23 Nov 2021
PointMixer: MLP-Mixer for Point Cloud Understanding Jaesung Choe Chunghyun Park François Rameau Jaesik Park In So Kweon 3DPC 39 98 0 22 Nov 2021
Mesa: A Memory-saving Training Framework for Transformers Zizheng Pan Peng Chen Haoyu He Jing Liu Jianfei Cai Bohan Zhuang 23 20 0 22 Nov 2021
Swin Transformer V2: Scaling Up Capacity and Resolution Ze Liu Han Hu Yutong Lin Zhuliang Yao Zhenda Xie ... Yue Cao Zheng-Wei Zhang Li Dong Furu Wei B. Guo ViT 52 1,746 0 18 Nov 2021
Restormer: Efficient Transformer for High-Resolution Image Restoration Syed Waqas Zamir Aditya Arora Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 37 2,123 0 18 Nov 2021
TransMix: Attend to Mix for Vision Transformers Jieneng Chen Shuyang Sun Ju He Philip H. S. Torr Alan Yuille S. Bai ViT 20 103 0 18 Nov 2021
Dynamically pruning segformer for efficient semantic segmentation Haoli Bai Hongda Mao D. Nair 25 20 0 18 Nov 2021
Searching for TrioNet: Combining Convolution with Local and Global Self-Attention Huaijin Pi Huiyu Wang Yingwei Li Zizhang Li Alan Yuille ViT 21 3 0 15 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 71 330 0 11 Nov 2021
Are Transformers More Robust Than CNNs? Yutong Bai Jieru Mei Alan Yuille Cihang Xie ViT AAML 192 257 0 10 Nov 2021
Are we ready for a new paradigm shift? A Survey on Visual Deep MLP Ruiyang Liu Yinghui Li Li Tao Dun Liang Haitao Zheng 85 97 0 07 Nov 2021
Relational Self-Attention: What's Missing in Attention for Video Understanding Manjin Kim Heeseung Kwon Chunyu Wang Suha Kwak Minsu Cho ViT 27 28 0 02 Nov 2021
Blending Anti-Aliasing into Vision Transformer Shengju Qian Hao Shao Yi Zhu Mu Li Jiaya Jia 23 20 0 28 Oct 2021
MVT: Multi-view Vision Transformer for 3D Object Recognition Shuo Chen Tan Yu Ping Li ViT 34 43 0 25 Oct 2021
SOFT: Softmax-free Transformer with Linear Complexity Jiachen Lu Jinghan Yao Junge Zhang Martin Danelljan Hang Xu Weiguo Gao Chunjing Xu Thomas B. Schon Li Zhang 18 161 0 22 Oct 2021
SSAST: Self-Supervised Audio Spectrogram Transformer Yuan Gong Cheng-I Jeff Lai Yu-An Chung James R. Glass ViT 32 268 0 19 Oct 2021
HRFormer: High-Resolution Transformer for Dense Prediction Yuhui Yuan Rao Fu Lang Huang Weihong Lin Chao Zhang Xilin Chen Jingdong Wang ViT 29 227 0 18 Oct 2021
ASFormer: Transformer for Action Segmentation Fangqiu Yi Hongyu Wen Tingting Jiang ViT 73 172 0 16 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 30 39 0 10 Oct 2021
Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs Philipp Benz Soomin Ham Chaoning Zhang Adil Karjauv In So Kweon AAML ViT 41 78 0 06 Oct 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 212 1,212 0 05 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 212 487 0 01 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 114 20 0 29 Sep 2021
PnP-DETR: Towards Efficient Visual Analysis with Transformers Tao Wang Li Yuan Yunpeng Chen Jiashi Feng Shuicheng Yan ViT 24 82 0 15 Sep 2021
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation Tongkun Xu Weihua Chen Pichao Wang Fan Wang Hao Li R. L. Jin ViT 56 215 0 13 Sep 2021
Compute and Energy Consumption Trends in Deep Learning Inference Radosvet Desislavov Fernando Martínez-Plumed José Hernández Orallo 35 113 0 12 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 19 41 0 08 Sep 2021
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting R. Liu Hanming Deng Yangyi Huang Xiaoyu Shi Lewei Lu Wenxiu Sun Xiaogang Wang Jifeng Dai Hongsheng Li ViT 22 124 0 07 Sep 2021
Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation Ziniu Wan Zhengjia Li Maoqing Tian Jianbo Liu Shuai Yi Hongsheng Li 3DH 32 80 0 06 Sep 2021
Hire-MLP: Vision MLP via Hierarchical Rearrangement Jianyuan Guo Yehui Tang Kai Han Xinghao Chen Han Wu Chao Xu Chang Xu Yunhe Wang 43 105 0 30 Aug 2021
Do Vision Transformers See Like Convolutional Neural Networks? M. Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang Alexey Dosovitskiy ViT 52 924 0 19 Aug 2021
Causal Attention for Unbiased Visual Recognition Tan Wang Chan Zhou Qianru Sun Hanwang Zhang OOD CML 32 108 0 19 Aug 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 177 476 0 12 Aug 2021
RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality? Yuki Tatsunami Masato Taki 24 12 0 09 Aug 2021
TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network Zhengyi Liu Yuan Wang Zhengzheng Tu Yun Xiao Bin Tang ViT 32 142 0 09 Aug 2021
Armour: Generalizable Compact Self-Attention for Vision Transformers Lingchuan Meng ViT 19 3 0 03 Aug 2021
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer Yifan Xu Zhijie Zhang Mengdan Zhang Kekai Sheng Ke Li Weiming Dong Liqing Zhang Changsheng Xu Xing Sun ViT 29 201 0 03 Aug 2021
Congested Crowd Instance Localization with Dilated Convolutional Swin Transformer Junyuan Gao Maoguo Gong Xuelong Li ViT 19 46 0 02 Aug 2021
CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention Wenxiao Wang Lulian Yao Long Chen Binbin Lin Deng Cai Xiaofei He Wei Liu 32 256 0 31 Jul 2021
DPT: Deformable Patch-based Transformer for Visual Recognition Zhiyang Chen Yousong Zhu Chaoyang Zhao Guosheng Hu Wei Zeng Jinqiao Wang Ming Tang ViT 16 98 0 30 Jul 2021
Query2Label: A Simple Transformer Way to Multi-Label Classification Shilong Liu Lei Zhang Xiao Yang Hang Su Jun Zhu 18 187 0 22 Jul 2021
CycleMLP: A MLP-like Architecture for Dense Prediction Shoufa Chen Enze Xie Chongjian Ge Runjian Chen Ding Liang Ping Luo 19 231 0 21 Jul 2021
All the attention you need: Global-local, spatial-channel attention for image retrieval Chull Hwan Song Hye Joo Han Yannis Avrithis 11 39 0 16 Jul 2021
A Comparative Study of Deep Learning Classification Methods on a Small Environmental Microorganism Image Dataset (EMDS-6): from Convolutional Neural Networks to Visual Transformers Penghui Zhao Chen Li M. Rahaman Hao Xu Hechen Yang Hongzan Sun Tao Jiang M. Grzegorzek VLM 24 39 0 16 Jul 2021