Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1504.00941
Cited By
v1
v2 (latest)
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
3 April 2015
Quoc V. Le
Navdeep Jaitly
Geoffrey E. Hinton
ODL
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"A Simple Way to Initialize Recurrent Networks of Rectified Linear Units"
50 / 353 papers shown
Stabilizing RNN Gradients through Pre-training
Luca Herranz-Celotti
Jean Rouat
241
1
0
23 Aug 2023
Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks
IEEE International Joint Conference on Neural Network (IJCNN), 2023
Ran Dou
José C. Príncipe
171
2
0
28 Jul 2023
Fading memory as inductive bias in residual recurrent networks
Neural Networks (Neural Netw.), 2023
I. Dubinin
Felix Effenberger
223
9
0
27 Jul 2023
Long Short-term Memory with Two-Compartment Spiking Neuron
Shimin Zhang
Qu Yang
Chenxiang Ma
Jibin Wu
Haizhou Li
Kay Chen Tan
171
9
0
14 Jul 2023
Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecasting
Adamantios Ntakaris
Moncef Gabbouj
Juho Kanniainen
AI4TS
209
2
0
17 Apr 2023
SMPConv: Self-moving Point Representations for Continuous Convolution
Computer Vision and Pattern Recognition (CVPR), 2023
Sanghyeon Kim
Eunbyung Park
3DPC
201
17
0
05 Apr 2023
Resurrecting Recurrent Neural Networks for Long Sequences
International Conference on Machine Learning (ICML), 2023
Antonio Orvieto
Samuel L. Smith
Albert Gu
Anushan Fernando
Çağlar Gülçehre
Razvan Pascanu
Soham De
497
420
0
11 Mar 2023
Convolutional unitary or orthogonal recurrent neural networks
M. Magnasco
124
2
0
14 Feb 2023
A Natural Bias for Language Generation Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Clara Meister
Wojciech Stokowiec
Tiago Pimentel
Lei Yu
Laura Rimell
A. Kuncoro
MILM
183
6
0
19 Dec 2022
State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
Cheng Wang
Carolin (Haas) Lawrence
Mathias Niepert
217
3
0
10 Dec 2022
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback
International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
N. Benjamin Erichson
Soon Hoe Lim
Michael W. Mahoney
309
8
0
01 Dec 2022
Exploring the Long-Term Generalization of Counting Behavior in RNNs
Nadine El-Naggar
Pranava Madhyastha
Tillman Weyde
81
6
0
29 Nov 2022
Lempel-Ziv Networks
Rebecca Saul
Mohammad Mahmudul Alam
John Hurwitz
Edward Raff
Tim Oates
James Holt
175
2
0
23 Nov 2022
On the biological plausibility of orthogonal initialisation for solving gradient instability in deep neural networks
Nikolay Manchev
Michael W. Spratling
ODL
122
1
0
27 Oct 2022
On Scrambling Phenomena for Randomly Initialized Recurrent Networks
Neural Information Processing Systems (NeurIPS), 2022
Vaggos Chatziafratis
Ioannis Panageas
Clayton Sanford
S. Stavroulakis
195
2
0
11 Oct 2022
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks
AAAI Conference on Artificial Intelligence (AAAI), 2022
Kentaro Ohno
Sekitoshi Kanai
Yasutoshi Ida
290
1
0
04 Oct 2022
Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022
Andrea Ceni
ODL
156
12
0
03 Oct 2022
An Embarrassingly Simple Approach for Intellectual Property Rights Protection on Recurrent Neural Networks
Zhi Qin Tan
H. P. Wong
Chee Seng Chan
217
3
0
03 Oct 2022
Efficient LSTM Training with Eligibility Traces
International Conference on Artificial Neural Networks (ICANN), 2022
Mitchell L. Hoyer
Shahram Eivazi
S. Otte
71
2
0
30 Sep 2022
Breaking Time Invariance: Assorted-Time Normalization for RNNs
Neural Processing Letters (NPL), 2022
Cole Pospisil
Vasily Zadorozhnyy
Qiang Ye
95
1
0
28 Sep 2022
Towards Large-Scale Small Object Detection: Survey and Benchmarks
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
Gong Cheng
Xiang Yuan
Xiwen Yao
Ke Yan
Qinghua Zeng
Xingxing Xie
Junwei Han
ObjD
421
529
0
28 Jul 2022
Privacy-Preserving Federated Recurrent Neural Networks
Proceedings on Privacy Enhancing Technologies (PoPETs), 2022
Sinem Sav
Abdulrahman Diaa
Apostolos Pyrgelis
Jean-Philippe Bossuat
Jean-Pierre Hubaux
FedML
248
10
0
28 Jul 2022
Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomes
Neural Information Processing Systems (NeurIPS), 2022
A. Sorokin
N. Buzun
Leonid Pugachev
Andrey Kravchenko
369
11
0
27 Jul 2022
Hybrid Facial Expression Recognition (FER2013) Model for Real-Time Emotion Classification and Prediction
O. Oguine
K. J. Oguine
Hashim Ibrahim Bisallah
Daniel Ofuani
CVBM
224
29
0
19 Jun 2022
RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
Shanghua Gao
Zhong-Yu Li
Qi Han
Ming-Ming Cheng
Liang Wang
307
42
0
14 Jun 2022
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
International Conference on Learning Representations (ICLR), 2022
Anand Subramoney
Khaleelulla Khan Nazeer
Mark Schöne
Christian Mayr
David Kappel
334
30
0
13 Jun 2022
Towards a General Purpose CNN for Long Range Dependencies in
N
N
N
D
David W. Romero
David M. Knigge
Albert Gu
Erik J. Bekkers
E. Gavves
Jakub M. Tomczak
Mark Hoogendoorn
237
25
0
07 Jun 2022
Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representations
International Conference on Learning Representations (ICLR), 2022
Ali Hummos
CLL
215
12
0
24 May 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs
AAAI Conference on Artificial Intelligence (AAAI), 2022
Fanchen Bu
D. Chang
173
7
0
12 May 2022
MemFHE: End-to-End Computing with Fully Homomorphic Encryption in Memory
ACM Transactions on Embedded Computing Systems (TECS), 2022
Saransh Gupta
Rosario Cammarota
Tajana Simunic
155
38
0
26 Apr 2022
Path Development Network with Finite-dimensional Lie Group Representation
Han Lou
Siran Li
Hao Ni
241
10
0
02 Apr 2022
What is the best RNN-cell structure for forecasting each time series behavior?
Expert systems with applications (ESWA), 2022
Rohaifa Khaldi
A. E. Afia
R. Chiheb
Siham Tabik
AI4TS
78
1
0
15 Mar 2022
NeuroView-RNN: It's About Time
Conference on Fairness, Accountability and Transparency (FAccT), 2022
C. Barberan
Sina Alemohammad
Naiming Liu
Randall Balestriero
Richard G. Baraniuk
AI4TS
HAI
193
2
0
23 Feb 2022
Semantic-based Data Augmentation for Math Word Problems
International Conference on Database Systems for Advanced Applications (DASFAA), 2022
Ai Li
Jiaqing Liang
Yanghua Xiao
AAML
203
7
0
07 Jan 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks
Neurocomputing (Neurocomputing), 2022
Lei Cheng
Ruslan Khalitov
Tong Yu
Zhirong Yang
218
41
0
06 Jan 2022
Target Propagation via Regularized Inversion
Vincent Roulet
Zaïd Harchaoui
BDL
AAML
257
5
0
02 Dec 2021
Gradients are Not All You Need
Luke Metz
C. Freeman
S. Schoenholz
Tal Kachman
239
102
0
10 Nov 2021
Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representation
International Conference on Big Knowledge (ICBK), 2021
Kentaro Ohno
Atsutoshi Kumagai
CLL
AI4CE
92
10
0
05 Nov 2021
Cortico-cerebellar networks as decoupling neural interfaces
Neural Information Processing Systems (NeurIPS), 2021
J. Pemberton
E. Boven
Richard Apps
Rui Ponte Costa
179
7
0
21 Oct 2021
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
David W. Romero
Robert-Jan Bruintjes
Jakub M. Tomczak
Erik J. Bekkers
Mark Hoogendoorn
Jan van Gemert
468
92
0
15 Oct 2021
How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies
Bao Wang
Hedi Xia
T. Nguyen
Stanley Osher
AI4CE
171
12
0
13 Oct 2021
Orthogonal Graph Neural Networks
AAAI Conference on Artificial Intelligence (AAAI), 2021
Kai Guo
Kaixiong Zhou
Helen Zhou
Yu Li
Yi Chang
Xin Wang
205
40
0
23 Sep 2021
Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing
Bing Han
Cheng Wang
Kaushik Roy
124
7
0
14 Sep 2021
CAN3D: Fast 3D Medical Image Segmentation via Compact Context Aggregation
Wei Dai
B. Woo
Siyu Liu
Matthew Marques
Craig B. Engstrom
P. Greer
Stuart Crozier
Jason Dowling
Shekhar S. Chandra
234
22
0
12 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks
K. Aoyama
H. Sawada
150
2
0
01 Sep 2021
Working Memory Connections for LSTM
Neural Networks (NN), 2021
Federico Landi
Lorenzo Baraldi
Marcella Cornia
Rita Cucchiara
KELM
144
230
0
31 Aug 2021
Dirichlet Energy Constrained Learning for Deep Graph Neural Networks
Kaixiong Zhou
Xiao Shi Huang
Daochen Zha
Rui Chen
Li Li
Soo-Hyun Choi
Helen Zhou
GNN
AI4CE
191
146
0
06 Jul 2021
Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN
Neural Networks (NN), 2021
Haowei Jiang
Fei-wei Qin
Jin Cao
Yong Peng
Yanli Shao
LRM
ODL
146
53
0
22 Jun 2021
RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks
L. Kozachkov
Michaela Ennis
Jean-Jacques E. Slotine
304
24
0
16 Jun 2021
A Lightweight and Gradient-Stable Neural Layer
Neural Networks (NN), 2021
Yueyao Yu
Yin Zhang
503
1
0
08 Jun 2021
Previous
1
2
3
4
5
6
7
8
Next
