On The Computational Complexity of Self-Attention

11 September 2022

Papers citing "On The Computational Complexity of Self-Attention"

19 / 19 papers shown

Title
Masked Sensory-Temporal Attention for Sensor Generalization in Quadruped Locomotion Dikai Liu Tianwei Zhang Jianxiong Yin Simon See 80 1 0 13 Mar 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 72 0 0 03 Mar 2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers Weikang Meng Yadan Luo Xin Li D. Jiang Zheng Zhang 46 0 0 25 Jan 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 48 11 0 03 Jan 2025
RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecasting Suhan Guo Jiahong Deng Yi Wei Hui Dou F. Shen Jian Zhao AI4TS 50 0 0 31 Oct 2024
MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI Arya Tschand Arun Tejusve Raghunath Rajan S. Idgunji Anirban Ghosh J. Holleman ... Rowan Taubitz Sean Zhan Scott Wasson David Kanter Vijay Janapa Reddi 55 2 0 15 Oct 2024
MambaRecon: MRI Reconstruction with Structured State Space Models Yilmaz Korkmaz V. Patel Mamba 26 2 0 19 Sep 2024
T1-contrast Enhanced MRI Generation from Multi-parametric MRI for Glioma Patients with Latent Tumor Conditioning Zach Eidex Mojtaba Safari Richard L. J. Qiu D. Yu Hui-Kuo Shu H. Mao Xiaofeng Yang MedIm 27 1 0 03 Sep 2024
DeMansia: Mamba Never Forgets Any Tokens Ricky Fang Mamba 14 0 0 04 Aug 2024
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 50 2 0 07 Jun 2024
EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation Yuda Wang Xuxin He Shengxin Zhu Mamba 16 8 0 04 Jun 2024
QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks Chenhui Xu Fuxun Yu Zirui Xu Chenchen Liu Jinjun Xiong Xiang Chen 15 4 0 29 Nov 2023
Language Model Tokenizers Introduce Unfairness Between Languages Aleksandar Petrov Emanuele La Malfa Philip H. S. Torr Adel Bibi 16 96 0 17 May 2023
A Contrastive Learning Scheme with Transformer Innate Patches S. Jyhne Per-Arne Andersen Morten Goodwin Olsen ViT 8 0 0 26 Mar 2023
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 6 24 0 17 Jun 2022
Deciphering the Language of Nature: A transformer-based language model for deleterious mutations in proteins Theodore Jiang Li Fang Kai Wang MedIm 17 17 0 27 Oct 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 216 2,404 0 04 Jan 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 246 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020