Fast-Slow Recurrent Neural Networks

24 May 2017

Papers citing "Fast-Slow Recurrent Neural Networks"

44 / 44 papers shown

Title
Recurrent neural networks: vanishing and exploding gradients are not the end of the story Nicolas Zucchet Antonio Orvieto ODL AAML 45 9 0 31 May 2024
A Quantitative Review on Language Model Efficiency Research Meng Jiang Hy Dang Lingbo Tong 30 0 0 28 May 2023
Long-horizon video prediction using a dynamic latent hierarchy Alexey Zakharov Qinghai Guo Z. Fountas 31 4 0 29 Dec 2022
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki Edoardo Ponti 22 42 0 17 Nov 2022
FLOWGEN: Fast and slow graph generation Aman Madaan Yiming Yang AI4CE 24 3 0 15 Jul 2022
An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications Kazuki Hayashi S. Sakaino T. Tsuji 14 14 0 16 Mar 2022
Variational Predictive Routing with Nested Subjective Timescales Alexey Zakharov Qinghai Guo Z. Fountas BDL AI4TS 43 9 0 21 Oct 2021
Clockwork Variational Autoencoders Vaibhav Saxena Jimmy Ba Danijar Hafner VGen DRL 26 49 0 18 Feb 2021
RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm Yun Yue Ming Li Venkatesh Saligrama Ziming Zhang 11 4 0 12 Oct 2020
Demystifying Deep Learning in Predictive Spatio-Temporal Analytics: An Information-Theoretic Framework Qi Tan Yang Liu Jiming Liu AI4TS 25 8 0 14 Sep 2020
A New Training Pipeline for an Improved Neural Transducer Albert Zeyer André Merboldt Ralf Schluter Hermann Ney AI4TS MedIm 22 52 0 19 May 2020
Neuronal Sequence Models for Bayesian Online Inference Sascha Frölich D. Marković S. Kiebel 13 9 0 02 Apr 2020
AutoFoley: Artificial Synthesis of Synchronized Sound Tracks for Silent Videos with Deep Learning Sanchita Ghose John J. Prevost VGen 8 46 0 21 Feb 2020
Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges T. H. Le Hao Chen Muhammad Ali Babar VLM 64 152 0 13 Feb 2020
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection Guangxiang Zhao Junyang Lin Zhiyuan Zhang Xuancheng Ren Qi Su Xu Sun 22 108 0 25 Dec 2019
Thick-Net: Parallel Network Structure for Sequential Modeling Yu-Xuan Li Jin-Yuan Liu Liang Li Xiang Guan 11 0 0 19 Nov 2019
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory Kyle E. Helfrich Qiang Ye 16 6 0 18 Nov 2019
Multi-Zone Unit for Recurrent Neural Networks Fandong Meng Jinchao Zhang Yang Liu Jie Zhou AI4CE 16 2 0 17 Nov 2019
Deep Independently Recurrent Neural Network (IndRNN) Shuai Li Wanqing Li Chris Cook Yanbo Gao 23 50 0 11 Oct 2019
Meta-Learning with Warped Gradient Descent Sebastian Flennerhag Andrei A. Rusu Razvan Pascanu Francesco Visin Hujun Yin R. Hadsell 8 209 0 30 Aug 2019
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients? Anil Kag Ziming Zhang Venkatesh Saligrama 21 8 0 22 Aug 2019
Augmenting Self-attention with Persistent Memory Sainbayar Sukhbaatar Edouard Grave Guillaume Lample Hervé Jégou Armand Joulin RALM KELM 21 135 0 02 Jul 2019
Multiplicative Models for Recurrent Language Modeling Diego Maupomé Marie-Jean Meurs KELM 11 1 0 30 Jun 2019
ARMIN: Towards a More Efficient and Light-weight Recurrent Memory Network Zhangheng Li Jia-Xing Zhong Jingjia Huang Tao Zhang Thomas H. Li Ge Li 19 2 0 28 Jun 2019
Dynamic Evaluation of Transformer Language Models Ben Krause Emmanuel Kahembwe Iain Murray Steve Renals 21 42 0 17 Apr 2019
DA-LSTM: A Long Short-Term Memory with Depth Adaptive to Non-uniform Information Flow in Sequential Data Yifeng Zhang Ka-Ho Chow Shueng-Han Gary Chan AI4TS 28 2 0 18 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 38 3,674 0 09 Jan 2019
Long Short-Term Memory with Dynamic Skip Connections Tao Gui Qi Zhang Lujun Zhao Y. Lin Minlong Peng Jingjing Gong Xuanjing Huang 35 27 0 09 Nov 2018
Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series Qiang Zhang Kyle Birkeland Yaodong Yang Y. Liu 30 9 0 08 Nov 2018
Trellis Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun 25 145 0 15 Oct 2018
Cell-aware Stacked LSTMs for Modeling Sentences Jihun Choi Taeuk Kim Sang-goo Lee AI4TS 19 4 0 07 Sep 2018
Improved Language Modeling by Decoding the Past Siddhartha Brahma BDL AI4TS 14 6 0 14 Aug 2018
Character-Level Language Modeling with Deeper Self-Attention Rami Al-Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones 24 386 0 09 Aug 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 24 170 0 22 Mar 2018
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN Shuai Li W. Li Chris Cook Ce Zhu Yanbo Gao 19 719 0 13 Mar 2018
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes Cedric De Boom Thomas Demeester Bart Dhoedt 10 8 0 02 Jan 2018
Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He Shaobing Gao Liang Xiao Daxue Liu Hangen He David Barber AIMat 37 64 0 05 Nov 2017
Rotational Unit of Memory Rumen Dangovski L. Jing Marin Soljacic 16 7 0 26 Oct 2017
Dynamic Evaluation of Neural Sequence Models Ben Krause Emmanuel Kahembwe Iain Murray Steve Renals 30 133 0 21 Sep 2017
Language Modeling with Highway LSTM Gakuto Kurata Bhuvana Ramabhadran G. Saon A. Sethy AI4TS 21 38 0 19 Sep 2017
Simple Recurrent Units for Highly Parallelizable Recurrence Tao Lei Yu Zhang Sida I. Wang Huijing Dai Yoav Artzi LRM 44 271 0 08 Sep 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,327 0 05 Nov 2016
Surprisal-Driven Zoneout K. Rocki Tomasz Kornuta Tegan Maharaj 29 8 0 24 Oct 2016
Multiplicative LSTM for sequence modelling Ben Krause Liang Lu Iain Murray Steve Renals 35 208 0 26 Sep 2016