An Attention Free Transformer

28 May 2021

Papers citing "An Attention Free Transformer"

23 / 23 papers shown

Title
Multi-View Learning with Context-Guided Receptance for Image Denoising Binghong Chen Tingting Chai Wei Jiang Yuanrong Xu Guanglu Zhou Xiangqian Wu 48 0 0 05 May 2025
Hadamard product in deep learning: Introduction, Advances and Challenges Grigorios G. Chrysos Yongtao Wu Razvan Pascanu Philip Torr V. Cevher AAML 98 0 0 17 Apr 2025
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Pingyi Chen Zhongyi Shui Chenglu Zhu Lin Yang MedIm 34 4 0 18 Oct 2024
Joint Fine-tuning and Conversion of Pretrained Speech and Language Models towards Linear Complexity Mutian He Philip N. Garner 80 0 0 09 Oct 2024
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective Jinhao Li Jiaming Xu Shan Huang Yonghua Chen Wen Li ... Jiayi Pan Li Ding Hao Zhou Yu Wang Guohao Dai 62 15 0 06 Oct 2024
Instance-Conditioned Adaptation for Large-scale Generalization of Neural Combinatorial Optimization Changliang Zhou Xi Lin Zhenkun Wang Xialiang Tong Mingxuan Yuan Qingfu Zhang 48 6 0 03 May 2024
DiJiang: Efficient Large Language Models through Compact Kernelization Hanting Chen Zhicheng Liu Xutao Wang Yuchuan Tian Yunhe Wang VLM 24 5 0 29 Mar 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 23 1 0 18 Dec 2023
Vision Transformers for Mobile Applications: A Short Survey Nahid Alam Steven Kolawole S. Sethi Nishant Bansali Karina Nguyen ViT 18 3 0 30 May 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 76 556 0 22 May 2023
Application of Transformers for Nonlinear Channel Compensation in Optical Systems Behnam Behinaein Hamgini H. Najafi Ali Bakhshali Zhuhong Zhang 19 1 0 25 Apr 2023
AutoFocusFormer: Image Segmentation off the Grid Chen Ziwen K. Patnaik Shuangfei Zhai Alvin Wan Zhile Ren A. Schwing Alex Colburn Li Fuxin 17 9 0 24 Apr 2023
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan ViT 37 151 0 24 Mar 2023
A Close Look at Spatial Modeling: From Attention to Convolution Xu Ma Huan Wang Can Qin Kunpeng Li Xing Zhao Jie Fu Yun Fu ViT 3DPC 17 11 0 23 Dec 2022
FsaNet: Frequency Self-attention for Semantic Segmentation Fengyu Zhang Ashkan Panahi Guangjun Gao AI4TS 23 28 0 28 Nov 2022
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 50 105 0 30 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 58 2,020 0 27 May 2022
SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya Michael S. Ryoo 25 6 0 26 Nov 2021
ResViT: Residual vision transformers for multi-modal medical image synthesis Onat Dalmaz Mahmut Yurt Tolga Çukur ViT MedIm 32 338 0 30 Jun 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 271 2,603 0 04 May 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 269 179 0 17 Feb 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 243 579 0 12 Mar 2020