ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1709.04057
  4. Cited By
Parallelizing Linear Recurrent Neural Nets Over Sequence Length
v1v2 (latest)

Parallelizing Linear Recurrent Neural Nets Over Sequence Length

12 September 2017
Eric Martin
Chris Cundy
ArXiv (abs)PDFHTML

Papers citing "Parallelizing Linear Recurrent Neural Nets Over Sequence Length"

50 / 52 papers shown
Selective Rotary Position Embedding
Selective Rotary Position Embedding
Sajad Movahedi
Timur Carstensen
Arshia Afzal
Frank Hutter
Antonio Orvieto
Volkan Cevher
291
0
0
21 Nov 2025
Misaligned by Design: Incentive Failures in Machine Learning
Misaligned by Design: Incentive Failures in Machine Learning
David Autor
Andrew Caplin
Daniel Martin
Philip Marx
109
0
0
10 Nov 2025
MossNet: Mixture of State-Space Experts is a Multi-Head Attention
MossNet: Mixture of State-Space Experts is a Multi-Head Attention
Shikhar Tuli
James Smith
Haris Jeelani
Chi-Heng Lin
Abhishek Patel
Vasili Ramanishka
Yen-Chang Hsu
Hongxia Jin
MoE
269
0
0
30 Oct 2025
TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
Vladyslav Moroshan
Julien N. Siems
Arber Zela
Timur Carstensen
Frank Hutter
AI4TSAI4CE
211
0
0
29 Oct 2025
SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism
SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism
Reda Marzouk
Shahaf Bassan
Guy Katz
FAtt
260
2
0
24 Oct 2025
Similarity-Aware Selective State-Space Modeling for Semantic Correspondence
Similarity-Aware Selective State-Space Modeling for Semantic Correspondence
Seungwook Kim
Minsu Cho
Mamba
200
0
0
29 Sep 2025
Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models
Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models
Aleksandar Terzić
Nicolas Menet
Michael Hersche
Thomas Hofmann
Abbas Rahimi
164
0
0
26 Sep 2025
A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systems
A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systems
Xavier Gonzalez
E. Kelly Buchanan
Hyun Dong Lee
Jerry W. Liu
Ke Alexander Wang
D. Zoltowski
Christopher Ré
Scott W. Linderman
111
2
0
26 Sep 2025
Elucidating the Design Space of Decay in Linear Attention
Elucidating the Design Space of Decay in Linear Attention
Zhen Qin
Xuyang Shen
Yiran Zhong
100
1
0
05 Sep 2025
Revisiting associative recall in modern recurrent models
Revisiting associative recall in modern recurrent models
Destiny Okpekpe
Antonio Orvieto
124
0
0
26 Aug 2025
Predictability Enables Parallelization of Nonlinear State Space Models
Predictability Enables Parallelization of Nonlinear State Space Models
Xavier Gonzalez
Leo Kozachkov
D. Zoltowski
Kenneth L. Clarkson
Scott W. Linderman
194
3
0
22 Aug 2025
Fast weight programming and linear transformers: from machine learning to neurobiology
Fast weight programming and linear transformers: from machine learning to neurobiology
Kazuki Irie
Samuel J. Gershman
170
0
0
11 Aug 2025
Prototype-Driven Structure Synergy Network for Remote Sensing Images Segmentation
Prototype-Driven Structure Synergy Network for Remote Sensing Images SegmentationIEEE Transactions on Geoscience and Remote Sensing (IEEE TGRS), 2025
Junyi Wang
Jinjiang Li
Guodong Fan
Yakun Ju
Xiang Fang
Alex C. Kot
139
2
0
06 Aug 2025
Knowing When to Quit: Probabilistic Early Exits for Speech Separation
Knowing When to Quit: Probabilistic Early Exits for Speech Separation
Kenny Falkær Olsen
Mads Østergaard
Karl Ulbæk
S. F. V. Nielsen
Rasmus Malik Høegh Lindrup
Bjørn Sand Jensen
Morten Mørup
UQCV
244
0
0
13 Jul 2025
RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling
RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence ModelingIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025
Xiuying Wei
Anunay Yadav
Razvan Pascanu
Çağlar Gülçehre
AI4TS
247
0
0
06 Jul 2025
Sequential-Parallel Duality in Prefix Scannable Models
Sequential-Parallel Duality in Prefix Scannable Models
Morris Yau
Sharut Gupta
Valerie Engelmayer
Kazuki Irie
Stefanie Jegelka
Jacob Andreas
371
5
0
12 Jun 2025
Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNs
Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNs
Manuel Brenner
G. Koppe
200
0
0
09 Jun 2025
How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation
How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation
Xin Lu
Yanyan Zhao
Si Wei
Shijin Wang
Bing Qin
Ting Liu
215
0
0
24 May 2025
Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models
Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models
Benjamin Walker
Lingyi Yang
Nicola Muca Cirone
C. Salvi
Terry Lyons
AI4TS
366
6
0
23 May 2025
Learning to Dissipate Energy in Oscillatory State-Space Models
Learning to Dissipate Energy in Oscillatory State-Space Models
Jared Boyer
T. Konstantin Rusch
Daniela Rus
276
1
0
17 May 2025
Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access
Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access
Xiang Hu
Jiaqi Leng
Jun Zhao
Kewei Tu
Wei Wu
Mamba
432
1
0
23 Apr 2025
DiffVox: A Differentiable Model for Capturing and Analysing Vocal Effects Distributions
DiffVox: A Differentiable Model for Capturing and Analysing Vocal Effects Distributions
Chin-Yun Yu
Marco A. Martínez-Ramírez
Junghyun Koo
B. Hayes
Wei-Hsiang Liao
Gyorgy Fazekas
Yuki Mitsufuji
DiffM
265
2
0
20 Apr 2025
Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion
Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion
Qisai Liu
Zhanhong Jiang
Joshua R. Waite
Chao Liu
Aditya Balu
Soumik Sarkar
AI4TS
240
0
0
11 Apr 2025
Resona: Improving Context Copying in Linear Recurrence Models with Retrieval
Resona: Improving Context Copying in Linear Recurrence Models with Retrieval
Xinyu Wang
Linrui Ma
Jerry Huang
Peng Lu
Prasanna Parthasarathi
Xiao-Wen Chang
Boxing Chen
Yufei Cui
KELM
432
3
0
28 Mar 2025
Fixed-Point RNNs: Interpolating from Diagonal to Dense
Fixed-Point RNNs: Interpolating from Diagonal to Dense
Sajad Movahedi
Felix Sarnthein
Nicola Muca Cirone
Antonio Orvieto
427
5
0
13 Mar 2025
Towards Scalable and Stable Parallelization of Nonlinear RNNs
Towards Scalable and Stable Parallelization of Nonlinear RNNsNeural Information Processing Systems (NeurIPS), 2024
Xavier Gonzalez
Andrew Warrington
Jimmy T.H. Smith
Scott W. Linderman
598
24
0
17 Jan 2025
VMamba: Visual State Space Model
VMamba: Visual State Space ModelNeural Information Processing Systems (NeurIPS), 2024
Yue Liu
Yunjie Tian
Yuzhong Zhao
Hongtian Yu
Lingxi Xie
Yaowei Wang
Qixiang Ye
Jianbin Jiao
Yunfan Liu
Mamba
1.1K
1,522
0
31 Dec 2024
Multi-Agent Reinforcement Learning with Selective State-Space Models
Multi-Agent Reinforcement Learning with Selective State-Space ModelsAdaptive Agents and Multi-Agent Systems (AAMAS), 2024
Jemma Daniel
Ruan de Kock
Louay Ben Nessir
Sasha Abramowitz
Omayma Mahjoub
Wiem Khlifi
Claude Formanek
Arnu Pretorius
Mamba
315
2
0
25 Oct 2024
Oscillatory State-Space Models
Oscillatory State-Space ModelsInternational Conference on Learning Representations (ICLR), 2024
T. Konstantin Rusch
Daniela Rus
AI4TS
915
21
0
04 Oct 2024
Real-Time Recurrent Learning using Trace Units in Reinforcement Learning
Real-Time Recurrent Learning using Trace Units in Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2024
Esraa Elelimy
Adam White
Michael Bowling
Martha White
OffRL
350
5
0
02 Sep 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
Liliang Ren
Yang Liu
Yadong Lu
Haoran Pan
Chen Liang
Weizhu Chen
Mamba
368
111
0
11 Jun 2024
LongSSM: On the Length Extension of State-space Models in Language
  Modelling
LongSSM: On the Length Extension of State-space Models in Language Modelling
Shida Wang
248
4
0
04 Jun 2024
Mamba-R: Vision Mamba ALSO Needs Registers
Mamba-R: Vision Mamba ALSO Needs RegistersComputer Vision and Pattern Recognition (CVPR), 2024
Feng Wang
Jiahao Wang
Sucheng Ren
Guoyizhe Wei
J. Mei
Wei Shao
Yuyin Zhou
Yaoyao Liu
Cihang Xie
Mamba
336
24
0
23 May 2024
Does Transformer Interpretability Transfer to RNNs?
Does Transformer Interpretability Transfer to RNNs?
Gonccalo Paulo
Thomas Marshall
Nora Belrose
199
8
0
09 Apr 2024
Softmax Attention with Constant Cost per Token
Softmax Attention with Constant Cost per Token
Franz A. Heinsen
144
1
0
08 Apr 2024
Linear Attention Sequence Parallelism
Linear Attention Sequence Parallelism
Weigao Sun
Zhen Qin
Dong Li
Xuyang Shen
Yu Qiao
Yiran Zhong
378
5
0
03 Apr 2024
Theoretical Foundations of Deep Selective State-Space Models
Theoretical Foundations of Deep Selective State-Space Models
Nicola Muca Cirone
Antonio Orvieto
Benjamin Walker
C. Salvi
Terry Lyons
Mamba
624
56
0
29 Feb 2024
Investigating Recurrent Transformers with Dynamic Halt
Investigating Recurrent Transformers with Dynamic Halt
Jishnu Ray Chowdhury
Cornelia Caragea
535
3
0
01 Feb 2024
Gated Linear Attention Transformers with Hardware-Efficient Training
Gated Linear Attention Transformers with Hardware-Efficient Training
Aaron Courville
Bailin Wang
Songlin Yang
Yikang Shen
Yoon Kim
443
300
0
11 Dec 2023
Hierarchically Gated Recurrent Neural Network for Sequence Modeling
Hierarchically Gated Recurrent Neural Network for Sequence Modeling
Zhen Qin
Aaron Courville
Yiran Zhong
196
117
0
08 Nov 2023
RWKV: Reinventing RNNs for the Transformer Era
RWKV: Reinventing RNNs for the Transformer EraConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Bo Peng
Eric Alcaide
Quentin G. Anthony
Alon Albalak
Samuel Arcadinho
...
Qihang Zhao
P. Zhou
Qinghua Zhou
Jian Zhu
Rui-Jie Zhu
578
845
0
22 May 2023
Transformer Working Memory Enables Regular Language Reasoning and
  Natural Language Length Extrapolation
Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length ExtrapolationConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Ta-Chung Chi
Ting-Han Fan
Alexander I. Rudnicky
Peter J. Ramadge
LRM
147
15
0
05 May 2023
Parallel Spiking Neurons with High Efficiency and Ability to Learn
  Long-term Dependencies
Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term DependenciesNeural Information Processing Systems (NeurIPS), 2023
Wei Fang
Zhaofei Yu
Zhaokun Zhou
Ding Chen
Yanqing Chen
Zhengyu Ma
T. Masquelier
Yonghong Tian
336
69
0
25 Apr 2023
Resurrecting Recurrent Neural Networks for Long Sequences
Resurrecting Recurrent Neural Networks for Long SequencesInternational Conference on Machine Learning (ICML), 2023
Antonio Orvieto
Samuel L. Smith
Albert Gu
Anushan Fernando
Çağlar Gülçehre
Razvan Pascanu
Soham De
497
418
0
11 Mar 2023
Parallelizing Legendre Memory Unit Training
Parallelizing Legendre Memory Unit TrainingInternational Conference on Machine Learning (ICML), 2021
Narsimha Chilkuri
C. Eliasmith
203
44
0
22 Feb 2021
Sub-Linear Memory: How to Make Performers SLiM
Sub-Linear Memory: How to Make Performers SLiMNeural Information Processing Systems (NeurIPS), 2020
Valerii Likhosherstov
K. Choromanski
Jared Davis
Xingyou Song
Adrian Weller
231
21
0
21 Dec 2020
Learning to Reconstruct and Segment 3D Objects
Learning to Reconstruct and Segment 3D Objects
Bo Yang
3DPC
207
1
0
19 Oct 2020
Learning Efficient Representations of Mouse Movements to Predict User
  Attention
Learning Efficient Representations of Mouse Movements to Predict User AttentionAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2020
Ioannis Arapakis
Luis A. Leiva
HAI
149
35
0
30 May 2020
Tensor Networks for Probabilistic Sequence Modeling
Tensor Networks for Probabilistic Sequence Modeling
Jacob Miller
Guillaume Rabusseau
John Terilla
299
5
0
02 Mar 2020
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed
  Self-Feedback Improves Accuracy and Stability
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and Stability
Ziming Zhang
Anil Kag
Alan Sullivan
Venkatesh Saligrama
113
6
0
02 Mar 2019
12
Next