Improving the Efficiency of Transformers for Resource-Constrained
Devices

Improving the Efficiency of Transformers for Resource-Constrained Devices

30 June 2021

Ajay Balasubramaniam

Shabbir Marzban

Papers citing "Improving the Efficiency of Transformers for Resource-Constrained Devices"

11 / 11 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao W. Xu Shibiao Xu ViT 75 0 0 06 May 2025
MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices Mohammadali Shakerdargah Shan Lu Chao Gao Di Niu 70 0 0 20 Nov 2024
CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models Xiaojun Xiao Sen Shen Qiming Bao Hongfei Rong Kairui Liu Zhongsheng Wang J. Liu 13 2 0 31 Aug 2024
SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design Seokju Yun Youngmin Ro ViT 34 29 0 29 Jan 2024
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 35 3 0 18 Aug 2023
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention Xinyu Liu Houwen Peng Ningxin Zheng Yuqing Yang Han Hu Yixuan Yuan ViT 20 273 0 11 May 2023
Large Scale Time-Series Representation Learning via Simultaneous Low and High Frequency Feature Bootstrapping Vandan Gorade Azad Singh Deepak Mishra SSL AI4TS 9 7 0 24 Apr 2022
Transformers in Medical Imaging: A Survey Fahad Shamshad Salman Khan Syed Waqas Zamir Muhammad Haris Khan Munawar Hayat F. Khan H. Fu ViT LM&MA MedIm 106 660 0 24 Jan 2022
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 225 2,427 0 04 Jan 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 138 221 0 31 Dec 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 225 571 0 12 Sep 2019