FasterViT: Fast Vision Transformers with Hierarchical Attention

FasterViT: Fast Vision Transformers with Hierarchical Attention

9 June 2023

Ali Hatamizadeh

Pavlo Molchanov

Papers citing "FasterViT: Fast Vision Transformers with Hierarchical Attention"

16 / 16 papers shown

Title
Rethinking Boundary Detection in Deep Learning-Based Medical Image Segmentation Yi-Mou Lin Dong-Ming Zhang X. B. Fang Yufan Chen K.-T. Cheng Hao Chen 28 0 0 06 May 2025
HOTFormerLoc: Hierarchical Octree Transformer for Versatile Lidar Place Recognition Across Ground and Aerial Views Ethan Griffiths Maryam Haghighat Simon Denman Clinton Fookes Milad Ramezani 3DPC 57 0 0 11 Mar 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application Chuanyang Zheng ViT 67 0 0 26 Jan 2025
MambaVision: A Hybrid Mamba-Transformer Vision Backbone Ali Hatamizadeh Jan Kautz Mamba 38 56 0 10 Jul 2024
Vision Transformer with Sparse Scan Prior Qihang Fan Huaibo Huang Mingrui Chen Ran He ViT 36 5 0 22 May 2024
TurboViT: Generating Fast Vision Transformers via Generative Architecture Search Alexander Wong Saad Abbasi Saeejith Nair ViT 21 1 0 22 Aug 2023
Hydra Attention: Efficient Attention with Many Heads Daniel Bolya Cheng-Yang Fu Xiaoliang Dai Peizhao Zhang Judy Hoffman 99 76 0 15 Sep 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,434 0 11 Nov 2021
Token Pooling in Vision Transformers D. Marin Jen-Hao Rick Chang Anurag Ranjan Anish K. Prabhu Mohammad Rastegari Oncel Tuzel ViT 65 66 0 08 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 207 487 0 01 Oct 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 109 209 0 26 Apr 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,523 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,622 0 24 Feb 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 251 2,012 0 28 Jul 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 268 10,214 0 16 Nov 2016