DeepViT: Towards Deeper Vision Transformer

22 March 2021

Linjie Yang

Papers citing "DeepViT: Towards Deeper Vision Transformer"

50 / 253 papers shown

Title
An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention Yehjin Shin Jeongwhan Choi Hyowon Wi Noseong Park 38 29 0 16 Dec 2023
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 28 15 0 15 Dec 2023
Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost Haolin Qin Daquan Zhou Tingfa Xu Ziyang Bian Jianan Li 27 9 0 14 Dec 2023
Polynomial-based Self-Attention for Table Representation learning Jayoung Kim Yehjin Shin Jeongwhan Choi Hyowon Wi Noseong Park LMTD 19 2 0 12 Dec 2023
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 25 4 0 07 Dec 2023
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation Xuwei Xu Sen Wang Yudong Chen Yanping Zheng Zhewei Wei Jiajun Liu ViT 22 8 0 06 Nov 2023
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 24 14 0 02 Nov 2023
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 32 16 0 31 Oct 2023
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing Wei Dong Dawei Yan Zhijun Lin Peng Wang 19 21 0 10 Oct 2023
SlowFormer: Universal Adversarial Patch for Attack on Compute and Energy Efficiency of Inference Efficient Vision Transformers K. Navaneet Soroush Abbasi Koohpayegani Essam Sleiman Hamed Pirsiavash AAML ViT 13 1 0 04 Oct 2023
PixArt- $α$ : Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis Junsong Chen Jincheng Yu Chongjian Ge Lewei Yao Enze Xie ... Zhongdao Wang James T. Kwok Ping Luo Huchuan Lu Zhenguo Li DiffM 28 385 0 30 Sep 2023
Masked Image Residual Learning for Scaling Deeper Vision Transformers Guoxi Huang Hongtao Fu A. Bors 26 7 0 25 Sep 2023
Interpretability-Aware Vision Transformer Yao Qiang Chengyin Li Prashant Khanduri D. Zhu ViT 80 7 0 14 Sep 2023
Toward a Deeper Understanding: RetNet Viewed through Convolution Chenghao Li Chaoning Zhang ViT 35 7 0 11 Sep 2023
ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation Xian Lin Zengqiang Yan Xianbo Deng Chuansheng Zheng Li Yu ViT MedIm 8 25 0 09 Sep 2023
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 19 24 0 04 Sep 2023
Is visual explanation with Grad-CAM more reliable for deeper neural networks? a case study with automatic pneumothorax diagnosis Zirui Qiu H. Rivaz Yiming Xiao FAtt 6 4 0 29 Aug 2023
CS-Mixer: A Cross-Scale Vision MLP Model with Spatial-Channel Mixing Jianwei Cui David A. Araujo Suman Saha Md Faisal Kabir BDL 36 0 0 25 Aug 2023
SG-Former: Self-guided Transformer with Evolving Token Reallocation Sucheng Ren Xingyi Yang Songhua Liu Xinchao Wang ViT 27 40 0 23 Aug 2023
Multi-event Video-Text Retrieval Gengyuan Zhang Jisen Ren Jindong Gu Volker Tresp 19 13 0 22 Aug 2023
MGMAE: Motion Guided Masking for Video Masked Autoencoding Bingkun Huang Zhiyu Zhao Guozhen Zhang Yu Qiao Limin Wang 22 30 0 21 Aug 2023
Patch Is Not All You Need Chang-bo Li Jie M. Zhang Yang Wei Zhilong Ji Jinfeng Bai Shiguang Shan ViT 44 1 0 21 Aug 2023
Dataset Quantization Daquan Zhou Kaixin Wang Jianyang Gu Xiang Peng Dongze Lian Yifan Zhang Yang You Jiashi Feng DD 29 37 0 21 Aug 2023
Revisiting Vision Transformer from the View of Path Ensemble Shuning Chang Pichao Wang Haowen Luo Fan Wang Mike Zheng Shou ViT 27 3 0 12 Aug 2023
LEST: Large-scale LiDAR Semantic Segmentation with Transformer Chuanyu Luo Nuo Cheng Sikun Ma Han Li Xiaohan Li Shengguang Lei Pu Li 3DPC ViT 19 2 0 14 Jul 2023
X-MLP: A Patch Embedding-Free MLP Architecture for Vision Xinyue Wang Zhicheng Cai Chenglei Peng ViT 11 5 0 02 Jul 2023
Towards Deep Attention in Graph Neural Networks: Problems and Remedies Soo Yong Lee Fanchen Bu Jaemin Yoo Kijung Shin GNN 11 30 0 04 Jun 2023
HomE: Homography-Equivariant Video Representation Learning Anirudh Sriram Adrien Gaidon Jiajun Wu Juan Carlos Niebles L. Fei-Fei Ehsan Adeli SSL AI4TS 18 2 0 02 Jun 2023
Student-friendly Knowledge Distillation Mengyang Yuan Bo Lang Fengnan Quan 18 17 0 18 May 2023
TempEE: Temporal-Spatial Parallel Transformer for Radar Echo Extrapolation Beyond Auto-Regression Shengchao Chen Ting Shu Huani Zhao Guo Zhong Xunlai Chen 27 17 0 27 Apr 2023
Hint-Aug: Drawing Hints from Foundation Vision Transformers Towards Boosted Few-Shot Parameter-Efficient Tuning Zhongzhi Yu Shang Wu Y. Fu Shunyao Zhang Yingyan Lin 25 6 0 25 Apr 2023
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel Yanli Zhao Andrew Gu R. Varma Liangchen Luo Chien-chin Huang ... Bernard Nguyen Geeta Chauhan Y. Hao Ajit Mathews Shen Li FedML MoE 32 304 0 21 Apr 2023
LipsFormer: Introducing Lipschitz Continuity to Vision Transformers Xianbiao Qi Jianan Wang Yihao Chen Yukai Shi Lei Zhang 22 16 0 19 Apr 2023
DarSwin: Distortion Aware Radial Swin Transformer Akshay Athwale Ichrak Shili Émile Bergeron Arman Afrasiyabi Justin Lague Ola Ahmad Jean-François Lalonde 21 6 0 19 Apr 2023
DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning Enze Xie Lewei Yao Han Shi Zhili Liu Daquan Zhou Zhaoqiang Liu Jiawei Li Zhenguo Li 24 76 0 13 Apr 2023
SpectFormer: Frequency and Attention is what you need in a Vision Transformer Badri N. Patro Vinay P. Namboodiri Vijay Srinivas Agneeswaran ViT 22 47 0 13 Apr 2023
StageInteractor: Query-based Object Detector with Cross-stage Interaction Yao Teng Haisong Liu Sheng Guo Limin Wang ObjD 29 8 0 11 Apr 2023
Weakly Supervised Intracranial Hemorrhage Segmentation using Head-Wise Gradient-Infused Self-Attention Maps from a Swin Transformer in Categorical Learning Amir Rasoulian Soorena Salari Yiming Xiao 6 8 0 11 Apr 2023
ViT-Calibrator: Decision Stream Calibration for Vision Transformer Lin Chen Zhijie Jia Tian Qiu Lechao Cheng Jie Lei Zunlei Feng Min-Gyoo Song 19 1 0 10 Apr 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention Mingyu Ding Yikang Shen Lijie Fan Zhenfang Chen Z. Chen Ping Luo J. Tenenbaum Chuang Gan ViT 77 14 0 06 Apr 2023
SVT: Supertoken Video Transformer for Efficient Video Understanding Chen-Ming Pan Rui Hou Hanchao Yu Qifan Wang Senem Velipasalar Madian Khabsa ViT 21 0 0 01 Apr 2023
PEOPL: Characterizing Privately Encoded Open Datasets with Public Labels H. Esfahanizadeh Adam Yala Rafael G. L. DÓliveira Andrea J. D. Jaba Victor Quach ... Tommi Jaakkola Vinod Vaikuntanathan M. Ghobadi Regina Barzilay Muriel Médard 17 0 0 31 Mar 2023
Zero-guidance Segmentation Using Zero Segment Labels Pitchaporn Rewatbowornwong Nattanat Chatthee E. Chuangsuwanich Supasorn Suwajanakorn VLM 25 11 0 23 Mar 2023
Robustifying Token Attention for Vision Transformers Yong Guo David Stutz Bernt Schiele ViT 14 24 0 20 Mar 2023
Scene Graph Based Fusion Network For Image-Text Retrieval Guoliang Wang Yanlei Shang Yongzhe Chen 24 1 0 20 Mar 2023
cito: An R package for training neural networks using torch Christian Amesoeder F. Hartig Maximilian Pichler 23 3 0 16 Mar 2023
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization Xingxuan Zhang Renzhe Xu Han Yu Hao Zou Peng Cui 16 39 0 03 Mar 2023
Are More Layers Beneficial to Graph Transformers? Haiteng Zhao Shuming Ma Dongdong Zhang Zhi-Hong Deng Furu Wei 27 12 0 01 Mar 2023
SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing Weidong Chen Xiaofen Xing Xiangmin Xu Jianxin Pang Lan Du 30 38 0 27 Feb 2023
Device Tuning for Multi-Task Large Model Penghao Jiang Xuanchen Hou Y. Zhou 11 0 0 21 Feb 2023