Hydra Attention: Efficient Attention with Many Heads

Hydra Attention: Efficient Attention with Many Heads

15 September 2022

Papers citing "Hydra Attention: Efficient Attention with Many Heads"

11 / 11 papers shown

Title
PolaFormer: Polarity-aware Linear Attention for Vision Transformers Weikang Meng Yadan Luo Xin Li D. Jiang Zheng Zhang 46 0 0 25 Jan 2025
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 28 0 0 12 Nov 2024
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers Enze Xie Junsong Chen Junyu Chen Han Cai Haotian Tang ... Zhekai Zhang Muyang Li Ligeng Zhu Y. Lu Song Han VLM 23 48 0 14 Oct 2024
Accelerating Transformers with Spectrum-Preserving Token Merging Hoai-Chau Tran D. M. Nguyen Duy M. Nguyen Trung Thanh Nguyen Ngan Le Pengtao Xie Daniel Sonntag James Y. Zou Binh T. Nguyen Mathias Niepert 32 8 0 25 May 2024
SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization Jialong Guo Xinghao Chen Yehui Tang Yunhe Wang ViT 44 9 0 19 May 2024
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights Moein Heidari Reza Azad Sina Ghorbani Kolahi René Arimond Leon Niggemeier ... Afshin Bozorgpour Ehsan Khodapanah Aghdam A. Kazerouni I. Hacihaliloglu Dorit Merhof 33 7 0 28 Mar 2024
Spike-driven Transformer Man Yao Jiakui Hu Zhaokun Zhou Liuliang Yuan Yonghong Tian Boxing Xu Guoqi Li 18 109 0 04 Jul 2023
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 184 1,148 0 05 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 106 16 0 29 Sep 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 167 462 0 12 Aug 2021