Unlocking the Secrets of Linear Complexity Sequence Model from A Unified
Perspective

Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective

27 May 2024

Zhen Qin

Stanley T. Birchfield

Richard I. Hartley

Papers citing "Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective"

7 / 7 papers shown

Title
Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts Weigao Sun Disen Lan Tong Zhu Xiaoye Qu Yu-Xi Cheng MoE 61 1 0 07 Mar 2025
MoM: Linear Sequence Modeling with Mixture-of-Memories Jusen Du Weigao Sun Disen Lan Jiaxi Hu Yu-Xi Cheng KELM 75 3 0 19 Feb 2025
Linear Attention Sequence Parallelism Weigao Sun Zhen Qin Dong Li Xuyang Shen Yu Qiao Yiran Zhong 68 2 0 03 Apr 2024
Zoology: Measuring and Improving Recall in Efficient Language Models Simran Arora Sabri Eyuboglu Aman Timalsina Isys Johnson Michael Poli James Zou Atri Rudra Christopher Ré 56 65 0 08 Dec 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 88 258 0 11 Mar 2023
Fine-Tuning Pre-trained Transformers into Decaying Fast Weights H. H. Mao 61 20 0 09 Oct 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021