Fastformer: Additive Attention Can Be All You Need

20 August 2021

Papers citing "Fastformer: Additive Attention Can Be All You Need"

50 / 51 papers shown

Title
Personalized News Recommendation with Multi-granularity Candidate-aware User Modeling Qiang Li Xinze Lin Shenghao Lv Faliang Huang X. Li 24 0 0 19 Apr 2025
Hadamard product in deep learning: Introduction, Advances and Challenges Grigorios G. Chrysos Yongtao Wu Razvan Pascanu Philip Torr V. Cevher AAML 96 0 0 17 Apr 2025
Generative Recommendation with Continuous-Token Diffusion Haohao Qu Wenqi Fan Shanru Lin DiffM 84 0 0 16 Apr 2025
Multi-Granularity Vision Fastformer with Fusion Mechanism for Skin Lesion Segmentation Xuanyu Liu Huiyun Yao Jinggui Gao Zhongyi Guo Xue Zhang Yulin Dong ViT MedIm 41 0 0 04 Apr 2025
Neural Attention: A Novel Mechanism for Enhanced Expressive Power in Transformer Models Andrew DiGiugno Ausif Mahmood 33 0 0 24 Feb 2025
In-context learning of evolving data streams with tabular foundational models Afonso Lourenço João Gama Eric P. Xing Goreti Marreiros 59 0 0 24 Feb 2025
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models Haoran Lian Junmin Chen Wei Huang Yizhe Xiong Wenping Hu ... Hui Chen Jianwei Niu Zijia Lin Fuzheng Zhang Di Zhang 81 0 0 10 Dec 2024
An Analysis of Linear Complexity Attention Substitutes with BEST-RQ Ryan Whetten Titouan Parcollet Adel Moumen Marco Dinarelli Yannick Esteve 22 0 0 04 Sep 2024
Linear-Complexity Self-Supervised Learning for Speech Processing Shucong Zhang Titouan Parcollet Rogier van Dalen Sourav Bhattacharya 28 1 0 18 Jul 2024
Breaking the Attention Bottleneck Kalle Hilsenbek 81 0 0 16 Jun 2024
Toward Short-Term Glucose Prediction Solely Based on CGM Time Series Ming Cheng Xingjian Diao Ziyi Zhou Yanjun Cui Wenjun Liu Shitong Cheng AI4TS 26 3 0 18 Apr 2024
CrossGP: Cross-Day Glucose Prediction Excluding Physiological Information Ziyi Zhou Ming Cheng Yanjun Cui Xingjian Diao Zhaorui Ma 26 3 0 16 Apr 2024
Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence Sung‐Jin Hong Seokju Cho Seungryong Kim Stephen Lin ViT 51 5 0 17 Mar 2024
Efficient generative adversarial networks using linear additive-attention Transformers Emilio Morales-Juarez Gibran Fuentes Pineda 29 3 0 17 Jan 2024
Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach Jinguo Cheng Ke Li Yuxuan Liang Lijun Sun Junchi Yan Yuankai Wu AI4TS 25 2 0 04 Dec 2023
HOT: Higher-Order Dynamic Graph Representation Learning with Efficient Transformers Maciej Besta Afonso Claudino Catarino Lukas Gianinazzi Nils Blach Piotr Nyczyk H. Niewiadomski Torsten Hoefler 30 6 0 30 Nov 2023
OAAFormer: Robust and Efficient Point Cloud Registration Through Overlapping-Aware Attention in Transformer Junjie Gao Qiujie Dong Ruian Wang Shuangmin Chen Shiqing Xin Changhe Tu Wenping Wang 16 1 0 15 Oct 2023
The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers Rickard Brannvall 14 0 0 03 Oct 2023
ConvFormer: Revisiting Transformer for Sequential User Modeling Hao Wang Jianxun Lian M. Wu Haoxuan Li Jiajun Fan Wanyue Xu Chaozhuo Li Xing Xie 17 3 0 05 Aug 2023
SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding Titouan Parcollet Rogier van Dalen Shucong Zhang S. Bhattacharya 16 6 0 12 Jul 2023
ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models Qijiong Liu Nuo Chen Tetsuya Sakai Xiao-Ming Wu 26 50 0 11 May 2023
XTab: Cross-table Pretraining for Tabular Transformers Bingzhao Zhu Xingjian Shi Nick Erickson Mu Li George Karypis Mahsa Shoaran LMTD 21 65 0 10 May 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 24 39 0 07 Apr 2023
Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR Rami Botros Anmol Gulati Tara N. Sainath K. Choromanski Ruoming Pang Trevor Strohman Weiran Wang Jiahui Yu MQ 10 3 0 31 Mar 2023
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang F. Khan ViT 35 83 0 27 Mar 2023
OAMatcher: An Overlapping Areas-based Network for Accurate Local Feature Matching Kun Dai Tao Xie K. Wang Zhiqiang Jiang Ruifeng Li Lijun Zhao 22 14 0 12 Feb 2023
Efficient Joint Learning for Clinical Named Entity Recognition and Relation Extraction Using Fourier Networks: A Use Case in Adverse Drug Events A. Yazdani D. Proios H. Rouhizadeh Douglas Teodoro 13 7 0 08 Feb 2023
DeepMatcher: A Deep Transformer-based Network for Robust and Accurate Local Feature Matching Tao Xie Kun Dai K. Wang Ruifeng Li Lijun Zhao ViT 22 44 0 08 Jan 2023
MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition Ziqi Gao Yuntao wang Jianguo Chen Junliang Xing Shwetak N. Patel Xin Liu Yuanchun Shi 31 3 0 14 Oct 2022
VLSNR:Vision-Linguistics Coordination Time Sequence-aware News Recommendation Songhao Han Wei Huang Xiaotian Luan Beihang University AI4TS 25 3 0 06 Oct 2022
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 50 105 0 30 Sep 2022
Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence Sunghwan Hong Seokju Cho Seung Wook Kim Stephen Lin 3DV 42 4 0 19 Sep 2022
User recommendation system based on MIND dataset Niran A. Abdulhussein Ahmed J. Obaid 13 2 0 06 Sep 2022
Uconv-Conformer: High Reduction of Input Sequence Length for End-to-End Speech Recognition A. Andrusenko R. Nasretdinov A. Romanenko 8 18 0 16 Aug 2022
Transformer-based Value Function Decomposition for Cooperative Multi-agent Reinforcement Learning in StarCraft Muhammad Junaid Khan Syed Hammad Ahmed G. Sukthankar 13 15 0 15 Aug 2022
Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation Sunghwan Hong Seokju Cho Jisu Nam Stephen Lin Seung Wook Kim ViT 19 122 0 22 Jul 2022
Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding Yifan Peng Siddharth Dalmia Ian Lane Shinji Watanabe 19 142 0 06 Jul 2022
Rethinking Query-Key Pairwise Interactions in Vision Transformers Cheng-rong Li Yangxin Liu 21 0 0 01 Jul 2022
Fair Comparison between Efficient Attentions Jiuk Hong Chaehyeon Lee Soyoun Bang Heechul Jung 17 1 0 01 Jun 2022
TransforMatcher: Match-to-Match Attention for Semantic Correspondence Seungwook Kim Juhong Min Minsu Cho ViT 38 32 0 23 May 2022
GRAM: Fast Fine-tuning of Pre-trained Language Models for Content-based Collaborative Filtering Yoonseok Yang Kyu Seok Kim Minsam Kim Juneyoung Park VLM 14 8 0 08 Apr 2022
A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification Dairui Liu Derek Greene Ruihai Dong 15 10 0 14 Mar 2022
Image Search with Text Feedback by Additive Attention Compositional Learning Yuxin Tian Shawn D. Newsam K. Boakye CoGe 19 11 0 08 Mar 2022
DCT-Former: Efficient Self-Attention with Discrete Cosine Transform Carmelo Scribano Giorgia Franchini M. Prato Marko Bertogna 10 21 0 02 Mar 2022
CATs++: Boosting Cost Aggregation with Convolutions and Transformers Seokju Cho Sunghwan Hong Seung Wook Kim ViT 19 34 0 14 Feb 2022
Boosting Robustness of Image Matting with Context Assembling and Strong Data Augmentation Yutong Dai Brian L. Price He Zhang Chunhua Shen 21 28 0 18 Jan 2022
Cost Aggregation Is All You Need for Few-Shot Segmentation Sunghwan Hong Seokju Cho Jisu Nam Seungryong Kim ViT 20 23 0 22 Dec 2021
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences Chao-Hong Tan Qian Chen Wen Wang Qinglin Zhang Siqi Zheng Zhenhua Ling ViT 11 11 0 06 Oct 2021
Audiomer: A Convolutional Transformer For Keyword Spotting Surya Kant Sahu Sai Mitheran Juhi Kamdar Meet Gandhi 18 8 0 21 Sep 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 2,009 0 28 Jul 2020