HiPPO: Recurrent Memory with Optimal Polynomial Projections

17 August 2020

Papers citing "HiPPO: Recurrent Memory with Optimal Polynomial Projections"

33 / 83 papers shown

Title
Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary Takashi Morita 16 3 0 31 Jan 2024
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization Shida Wang Qianxiao Li 22 13 0 24 Nov 2023
A Neural State-Space Model Approach to Efficient Speech Separation Chen Chen Chao-Han Huck Yang Kai Li Yuchen Hu Pin-Jui Ku Chng Eng Siong 31 11 0 26 May 2023
Neural Machine Translation for Code Generation K. Dharma Clayton T. Morrison 32 4 0 22 May 2023
State Spaces Aren't Enough: Machine Translation Needs Attention Ali Vardasbi Telmo Pires Robin M. Schmidt Stephan Peitz 19 9 0 25 Apr 2023
Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections E. Brouwer Rahul G. Krishnan AI4TS 17 0 0 03 Mar 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 16 52 0 13 Feb 2023
Scaling Up Computer Vision Neural Networks Using Fast Fourier Transform Siddharth Agrawal 15 0 0 02 Feb 2023
Diffusion-based Conditional ECG Generation with Structured State Space Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM 20 47 0 19 Jan 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 70 370 0 28 Dec 2022
Pretraining Without Attention Junxiong Wang J. Yan Albert Gu Alexander M. Rush 27 48 0 20 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 125 36 0 15 Dec 2022
Advancing the State-of-the-Art for ECG Analysis through Structured State Space Models Temesgen Mehari Nils Strodthoff 21 11 0 14 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 116 2,309 0 09 Nov 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 43 9 0 14 Oct 2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces Eric N. D. Nguyen Karan Goel Albert Gu Gordon W. Downs Preey Shah Tri Dao S. Baccus Christopher Ré VLM 22 38 0 12 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 104 95 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Mega: Moving Average Equipped Gated Attention Xuezhe Ma Chunting Zhou Xiang Kong Junxian He Liangke Gui Graham Neubig Jonathan May Luke Zettlemoyer 14 183 0 21 Sep 2022
fMRI-S4: learning short- and long-range dynamic fMRI dependencies using 1D Convolutions and State Space Models A. E. Gazzar R. Thomas G. Wingen 16 3 0 08 Aug 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 31 231 0 27 Jun 2022
On the Parameterization and Initialization of Diagonal State Space Models Albert Gu Ankit Gupta Karan Goel Christopher Ré 14 297 0 23 Jun 2022
Towards a General Purpose CNN for Long Range Dependencies in $N$ D David W. Romero David M. Knigge Albert Gu Erik J. Bekkers E. Gavves Jakub M. Tomczak Mark Hoogendoorn 16 19 0 07 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 63 2,024 0 27 May 2022
Long Movie Clip Classification with State-Space Video Models Md. Mohaiminul Islam Gedas Bertasius VLM 40 102 0 04 Apr 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 22 87 0 01 Apr 2022
Diagonal State Spaces are as Effective as Structured State Spaces Ankit Gupta Albert Gu Jonathan Berant 39 290 0 27 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 41 32 0 10 Mar 2022
Efficiently Modeling Long Sequences with Structured State Spaces Albert Gu Karan Goel Christopher Ré 52 1,654 0 31 Oct 2021
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn J. C. V. Gemert 80 82 0 15 Oct 2021
Rethinking Neural Operations for Diverse Tasks Nicholas Roberts M. Khodak Tri Dao Liam Li Christopher Ré Ameet Talwalkar AI4CE 36 22 0 29 Mar 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 243 580 0 12 Mar 2020
Recurrent Neural Networks for Multivariate Time Series with Missing Values Zhengping Che S. Purushotham Kyunghyun Cho David Sontag Yan Liu AI4TS 210 1,897 0 06 Jun 2016