SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization

19 May 2024

Papers citing "SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization"

9 / 9 papers shown

Title
Decision SpikeFormer: Spike-Driven Transformer for Decision Making Wei Huang Qinying Gu Nanyang Ye OffRL 23 1 0 04 Apr 2025
CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction Yuan Zhou Qingshan Xu Jiequan Cui Junbao Zhou Jing Zhang Richang Hong H. Zhang ViT 70 0 0 25 Nov 2024
LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment Ge Yang Changyi He J. Guo Jianyu Wu Yifu Ding Aishan Liu Haotong Qin Pengliang Ji Xianglong Liu MQ 28 0 0 28 Oct 2024
SLaNC: Static LayerNorm Calibration Mahsa Salmani Nikita Trukhanov I. Soloveychik MQ 23 0 0 14 Oct 2024
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 29 3 0 18 Aug 2023
BiFormer: Vision Transformer with Bi-Level Routing Attention Lei Zhu Xinjiang Wang Zhanghan Ke Wayne Zhang Rynson W. H. Lau 123 438 0 15 Mar 2023
Hydra Attention: Efficient Attention with Many Heads Daniel Bolya Cheng-Yang Fu Xiaoliang Dai Peizhao Zhang Judy Hoffman 96 75 0 15 Sep 2022
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021
RepVGG: Making VGG-style ConvNets Great Again Xiaohan Ding X. Zhang Ningning Ma Jungong Han Guiguang Ding Jian-jun Sun 117 1,484 0 11 Jan 2021