Linear Attention via Orthogonal Memory

Linear Attention via Orthogonal Memory

18 December 2023

Lingpeng Kong

Papers citing "Linear Attention via Orthogonal Memory"

5 / 5 papers shown

Title
ABC: Attention with Bounded-memory Control Hao Peng Jungo Kasai Nikolaos Pappas Dani Yogatama Zhaofeng Wu Lingpeng Kong Roy Schwartz Noah A. Smith 58 21 0 06 Oct 2021
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 161 3,799 0 14 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 214 7,687 0 17 Aug 2015