ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1504.00941
  4. Cited By
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
v1v2 (latest)

A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

3 April 2015
Quoc V. Le
Navdeep Jaitly
Geoffrey E. Hinton
    ODL
ArXiv (abs)PDFHTML

Papers citing "A Simple Way to Initialize Recurrent Networks of Rectified Linear Units"

50 / 353 papers shown
Recurrent Convolutions for Causal 3D CNNs
Recurrent Convolutions for Causal 3D CNNs
Gurkirt Singh
Fabio Cuzzolin
3DPC
131
0
0
17 Nov 2018
Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform
Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform
K. D. G. Maduranga
Kyle E. Helfrich
Qiang Ye
113
32
0
09 Nov 2018
Learning to Skip Ineffectual Recurrent Computations in LSTMs
Learning to Skip Ineffectual Recurrent Computations in LSTMsDesign, Automation and Test in Europe (DATE), 2018
A. Ardakani
Zhengyun Ji
W. Gross
79
16
0
09 Nov 2018
Counting in Language with RNNs
He Fun
Sergiy V. Bokhnyak
Francesco Saverio Zuppichini
119
0
0
29 Oct 2018
Bayesian Compression for Natural Language Processing
Bayesian Compression for Natural Language Processing
Nadezhda Chirkova
E. Lobacheva
Dmitry Vetrov
BDL
138
15
0
25 Oct 2018
h-detach: Modifying the LSTM Gradient Towards Better Optimization
h-detach: Modifying the LSTM Gradient Towards Better Optimization
Devansh Arpit
Bhargav Kanuparthi
Giancarlo Kerg
Nan Rosemary Ke
Alexia Jolicoeur-Martineau
Yoshua Bengio
311
36
0
06 Oct 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity
SNIP: Single-shot Network Pruning based on Connection Sensitivity
Namhoon Lee
Thalaiyasingam Ajanthan
Juil Sock
VLM
712
1,371
0
04 Oct 2018
Learning Recurrent Binary/Ternary Weights
Learning Recurrent Binary/Ternary WeightsInternational Conference on Learning Representations (ICLR), 2018
A. Ardakani
Zhengyun Ji
S. C. Smithson
B. Meyer
W. Gross
MQ
259
28
0
28 Sep 2018
A Deep Learning Spatiotemporal Prediction Framework for Mobile
  Crowdsourced Services
A Deep Learning Spatiotemporal Prediction Framework for Mobile Crowdsourced Services
Ahmed Ben Said
A. Erradi
A. G. Neiat
A. Bouguettaya
HAI
50
12
0
04 Sep 2018
Multimodal Language Analysis with Recurrent Multistage Fusion
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang
Liu Ziyin
Amir Zadeh
Louis-Philippe Morency
220
216
0
12 Aug 2018
3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks
3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks
Rongtian Ye
Fangyu Liu
Liqiang Zhang
MDE
90
51
0
05 Aug 2018
MCRM: Mother Compact Recurrent Memory
MCRM: Mother Compact Recurrent Memory
Abduallah A. Mohamed
Christian G. Claudel
135
1
0
04 Aug 2018
IGLOO: Slicing the Features Space to Represent Sequences
IGLOO: Slicing the Features Space to Represent Sequences
Vsevolod Sourkov
VLM
272
5
0
09 Jul 2018
Financial Trading as a Game: A Deep Reinforcement Learning Approach
Financial Trading as a Game: A Deep Reinforcement Learning Approach
Chien-Yi Huang
AIFin
107
78
0
08 Jul 2018
Sliced Recurrent Neural Networks
Sliced Recurrent Neural NetworksInternational Conference on Computational Linguistics (COLING), 2018
Zeping Yu
Gongshen Liu
117
45
0
06 Jul 2018
Beyond Backprop: Online Alternating Minimization with Auxiliary
  Variables
Beyond Backprop: Online Alternating Minimization with Auxiliary Variables
A. Choromańska
Benjamin Cowen
Yara Rizk
Ronny Luss
Mattia Rigotti
...
Brian Kingsbury
Paolo Diachille
V. Gurev
Ravi Tejwani
Djallel Bouneffouf
341
57
0
24 Jun 2018
Persistent Hidden States and Nonlinear Transformation for Long
  Short-Term Memory
Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory
Heeyoul Choi
99
15
0
22 Jun 2018
Detecting Cyberattacks in Industrial Control Systems Using Convolutional
  Neural Networks
Detecting Cyberattacks in Industrial Control Systems Using Convolutional Neural Networks
Moshe Kravchik
A. Shabtai
164
303
0
21 Jun 2018
Deep Recurrent Neural Network for Multi-target Filtering
Deep Recurrent Neural Network for Multi-target Filtering
Mehryar Emambakhsh
Alessandro Bay
E. Vazquez
117
6
0
18 Jun 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables
  Signal Propagation in Recurrent Neural Networks
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Minmin Chen
Jeffrey Pennington
S. Schoenholz
SyDaAI4CE
186
124
0
14 Jun 2018
Focused Hierarchical RNNs for Conditional Sequence Processing
Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke
Konrad Zolna
Alessandro Sordoni
Zhouhan Lin
Adam Trischler
Yoshua Bengio
Joelle Pineau
Laurent Charlin
C. Pal
AIMat
150
25
0
12 Jun 2018
On the Practical Computational Power of Finite Precision RNNs for
  Language Recognition
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
Gail Weiss
Yoav Goldberg
Eran Yahav
275
284
0
13 May 2018
Direction-aware Spatial Context Features for Shadow Detection and
  Removal
Direction-aware Spatial Context Features for Shadow Detection and Removal
Xiaowei Hu
Chi-Wing Fu
Lei Zhu
J. Qin
Pheng-Ann Heng
195
242
0
12 May 2018
A Taxonomy for Neural Memory Networks
A Taxonomy for Neural Memory Networks
Ying Ma
José C. Príncipe
226
25
0
01 May 2018
How Robust are Deep Neural Networks?
How Robust are Deep Neural Networks?
B. Sengupta
Karl J. Friston
OOD
127
37
0
30 Apr 2018
Deep Co-attention based Comparators For Relative Representation Learning
  in Person Re-identification
Deep Co-attention based Comparators For Relative Representation Learning in Person Re-identification
Lin Wu
Yang Wang
Junbin Gao
Dacheng Tao
123
21
0
30 Apr 2018
Deep Facial Expression Recognition: A Survey
Deep Facial Expression Recognition: A SurveyIEEE Transactions on Affective Computing (IEEE Trans. Affective Comput.), 2018
Shan Li
Weihong Deng
560
1,519
0
23 Apr 2018
Twin Regularization for online speech recognition
Twin Regularization for online speech recognition
Mirco Ravanelli
Dmitriy Serdyuk
Yoshua Bengio
114
16
0
15 Apr 2018
The unreasonable effectiveness of the forget gate
The unreasonable effectiveness of the forget gate
J. Westhuizen
Joan Lasenby
180
98
0
13 Apr 2018
QA4IE: A Question Answering based Framework for Information Extraction
QA4IE: A Question Answering based Framework for Information Extraction
Lin Qiu
Hao Zhou
Yanru Qu
Weinan Zhang
Suoheng Li
Shunlin Rong
Dongyu Ru
Lihua Qian
Kewei Tu
Yong Yu
202
21
0
10 Apr 2018
Regularizing RNNs for Caption Generation by Reconstructing The Past with
  The Present
Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present
Xinpeng Chen
Lin Ma
Wenhao Jiang
Jian Yao
Wen Liu
215
95
0
30 Mar 2018
Light Gated Recurrent Units for Speech Recognition
Light Gated Recurrent Units for Speech Recognition
Mirco Ravanelli
Philemon Brakel
M. Omologo
Yoshua Bengio
125
361
0
26 Mar 2018
Long short-term memory and learning-to-learn in networks of spiking
  neurons
Long short-term memory and learning-to-learn in networks of spiking neurons
G. Bellec
Darjan Salaj
Anand Subramoney
Robert Legenstein
Wolfgang Maass
497
542
0
26 Mar 2018
Stabilizing Gradients for Deep Neural Networks via Efficient SVD
  Parameterization
Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization
Jiong Zhang
Qi Lei
Inderjit S. Dhillon
210
118
0
25 Mar 2018
Can recurrent neural networks warp time?
Can recurrent neural networks warp time?
Corentin Tallec
Yann Ollivier
CLLAI4CE
342
151
0
23 Mar 2018
Learning Long Term Dependencies via Fourier Recurrent Units
Learning Long Term Dependencies via Fourier Recurrent Units
Jiong Zhang
Yibo Lin
Zhao Song
Inderjit S. Dhillon
149
44
0
17 Mar 2018
Independently Recurrent Neural Network (IndRNN): Building A Longer and
  Deeper RNN
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
Shuai Li
W. Li
Chris Cook
Ce Zhu
Yanbo Gao
369
797
0
13 Mar 2018
How to Start Training: The Effect of Initialization and Architecture
How to Start Training: The Effect of Initialization and Architecture
Boris Hanin
David Rolnick
257
273
0
05 Mar 2018
Beyond Context: Exploring Semantic Similarity for Tiny Face Detection
Beyond Context: Exploring Semantic Similarity for Tiny Face Detection
Yue Xi
Jiangbin Zheng
Xiangjian He
W. Jia
Hanhui Li
CVBM
105
3
0
05 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks
  for Sequence Modeling
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Shaojie Bai
J. Zico Kolter
V. Koltun
DRL
360
5,879
0
04 Mar 2018
Not All Samples Are Created Equal: Deep Learning with Importance
  Sampling
Not All Samples Are Created Equal: Deep Learning with Importance SamplingInternational Conference on Machine Learning (ICML), 2018
Angelos Katharopoulos
François Fleuret
379
604
0
02 Mar 2018
Learning Longer-term Dependencies in RNNs with Auxiliary Losses
Learning Longer-term Dependencies in RNNs with Auxiliary LossesInternational Conference on Machine Learning (ICML), 2018
Trieu H. Trinh
Andrew M. Dai
Thang Luong
Quoc V. Le
368
193
0
01 Mar 2018
Tensor Decomposition for Compressing Recurrent Neural Network
Tensor Decomposition for Compressing Recurrent Neural NetworkIEEE International Joint Conference on Neural Network (IJCNN), 2018
Andros Tjandra
S. Sakti
Satoshi Nakamura
116
54
0
28 Feb 2018
Deep Learning with a Rethinking Structure for Multi-label Classification
Deep Learning with a Rethinking Structure for Multi-label Classification
Yao-Yuan Yang
Yi-An Lin
Hong-Min Chu
Hsuan-Tien Lin
108
28
0
05 Feb 2018
Overcoming the vanishing gradient problem in plain recurrent networks
Overcoming the vanishing gradient problem in plain recurrent networks
Yuhuang Hu
Adrian E. G. Huber
Jithendar Anumula
Shih-Chii Liu
GNN
310
114
0
18 Jan 2018
Low-Shot Learning from Imaginary Data
Low-Shot Learning from Imaginary Data
Yu-Xiong Wang
Ross B. Girshick
M. Hebert
Bharath Hariharan
VLM
381
718
0
16 Jan 2018
Recent Advances in Recurrent Neural Networks
Recent Advances in Recurrent Neural Networks
Hojjat Salehinejad
Sharan Sankar
Joseph Barfett
E. Colak
S. Valaee
AI4TS
490
689
0
29 Dec 2017
Deep Learning for Distant Speech Recognition
Deep Learning for Distant Speech Recognition
Mirco Ravanelli
131
16
0
17 Dec 2017
Slim Embedding Layers for Recurrent Neural Language Models
Slim Embedding Layers for Recurrent Neural Language Models
Zhongliang Li
Raymond Kulhanek
Shaojun Wang
Yunxin Zhao
Shuang Wu
KELM
148
23
0
27 Nov 2017
Cortical microcircuits as gated-recurrent neural networks
Cortical microcircuits as gated-recurrent neural networks
Rui Ponte Costa
Yannis Assael
Brendan Shillingford
Nando de Freitas
T. Vogels
213
69
0
07 Nov 2017
Previous
12345678
Next