SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design

29 January 2024

Papers citing "SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design"

29 / 29 papers shown

Title
A Novel Hybrid Approach for Retinal Vessel Segmentation with Dynamic Long-Range Dependency and Multi-Scale Retinal Edge Fusion Enhancement Yihao Ouyang Xunheng Kuang Mengjia Xiong Zhida Wang Yuanquan Wang 36 0 0 18 Apr 2025
LSNet: See Large, Focus Small Ao Wang Hui Chen Zijia Lin J. Han Guiguang Ding 34 0 0 29 Mar 2025
Fraesormer: Learning Adaptive Sparse Transformer for Efficient Food Recognition Shun Zou Yi Zou Mingya Zhang Shipeng Luo Zhihao Chen Guangwei Gao ViT 43 0 0 15 Mar 2025
Enhancing Layer Attention Efficiency through Pruning Redundant Retrievals Hanze Li Xiande Huang 39 0 0 09 Mar 2025
Simpler Fast Vision Transformers with a Jumbo CLS Token A. Fuller Yousef Yassin Daniel G. Kyrollos Evan Shelhamer James R. Green 67 0 0 24 Feb 2025
MicroViT: A Vision Transformer with Low Complexity Self Attention for Edge Device Novendra Setyawan Chi-Chia Sun Mao-Hsiu Hsu W. Kuo Jun-Wei Hsieh ViT 41 1 0 09 Feb 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application Chuanyang Zheng ViT 67 0 0 26 Jan 2025
Rethinking Encoder-Decoder Flow Through Shared Structures Frederik Laboyrie M. K. Yucel Albert Saà-Garriga AI4CE 40 0 0 24 Jan 2025
RecConv: Efficient Recursive Convolutions for Multi-Frequency Representations Mingshu Zhao Yi Luo Yong Ouyang 28 0 0 27 Dec 2024
Residual Attention Single-Head Vision Transformer Network for Rolling Bearing Fault Diagnosis in Noisy Environments Songjiang Lai Tsun-hin Cheung Jiayi Zhao Kaiwen Xue Ka-Chun Fung K. Lam 77 0 0 27 Nov 2024
MobileMamba: Lightweight Multi-Receptive Visual Mamba Network Haoyang He J. Zhang Yuxuan Cai Hongxu Chen Xiaobin Hu Zhenye Gan Y. Wang Chengjie Wang Yunsheng Wu Lei Xie Mamba 77 3 0 24 Nov 2024
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality Sanghyeok Lee Joonmyung Choi Hyunwoo J. Kim 105 3 0 22 Nov 2024
Towards Better Multi-head Attention via Channel-wise Sample Permutation Shen Yuan Hongteng Xu 17 1 0 14 Oct 2024
Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies Yingqiang Gao Lukas Fischer Alexa Lintner Sarah Ebling 27 0 0 11 Oct 2024
HRVMamba: High-Resolution Visual State Space Model for Dense Prediction Hao Zhang Yongqiang Ma Wenqi Shao Ping Luo Nanning Zheng Kaipeng Zhang Mamba 20 1 0 04 Oct 2024
LowFormer: Hardware Efficient Design for Convolutional Transformer Backbones Moritz Nottebaum Matteo Dunnhofer C. Micheloni ViT 23 1 0 05 Sep 2024
RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization Mingshu Zhao Yi Luo Yong Ouyang 27 2 0 23 Jun 2024
Partial Large Kernel CNNs for Efficient Super-Resolution Dongheon Lee Seokju Yun Youngmin Ro SupR 26 0 0 18 Apr 2024
Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks Jierun Chen Shiu-hong Kao Hao He Weipeng Zhuo Song Wen Chul-Ho Lee Shueng-Han Gary Chan OOD 27 679 0 07 Mar 2023
Rethinking Mobile Block for Efficient Attention-based Models Jiangning Zhang Xiangtai Li Jian Li Liang Liu Zhucun Xue Boshen Zhang Zhe Jiang Tianxin Huang Yabiao Wang Chengjie Wang MQ 44 78 0 03 Jan 2023
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 189 1,148 0 05 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 172 462 0 12 Aug 2021
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 328 500 0 13 Jul 2021
Combiner: Full Attention Transformer with Sparse Computation Cost Hongyu Ren H. Dai Zihang Dai Mengjiao Yang J. Leskovec Dale Schuurmans Bo Dai 73 77 0 12 Jul 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
RepVGG: Making VGG-style ConvNets Great Again Xiaohan Ding X. Zhang Ningning Ma Jungong Han Guiguang Ding Jian-jun Sun 117 1,484 0 11 Jan 2021
Dynamic ReLU Yinpeng Chen Xiyang Dai Mengchen Liu Dongdong Chen Lu Yuan Zicheng Liu 158 157 0 22 Mar 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,214 0 17 Apr 2017