On orthogonality and learning recurrent networks with long term
dependencies

On orthogonality and learning recurrent networks with long term dependencies

31 January 2017

Eugene Vorontsov

Samuel Kadoury

Papers citing "On orthogonality and learning recurrent networks with long term dependencies"

18 / 18 papers shown

Title
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 64 0 0 28 Jan 2025
Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints Navid Shervani-Tabar Marzieh Alireza Mirhoseini Robert Rosenbaum AAML AI4CE 27 0 0 15 Aug 2024
Initializing Models with Larger Ones Zhiqiu Xu Yanjie Chen Kirill Vishniakov Yida Yin Zhiqiang Shen Trevor Darrell Lingjie Liu Zhuang Liu 20 17 0 30 Nov 2023
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold Jun Chen Haishan Ye Mengmeng Wang Tianxin Huang Guangwen Dai Ivor W.Tsang Y. Liu 6 9 0 21 Aug 2023
Fading memory as inductive bias in residual recurrent networks I. Dubinin Felix Effenberger 16 4 0 27 Jul 2023
Stabilising and accelerating light gated recurrent units for automatic speech recognition Adel Moumen Titouan Parcollet 16 3 0 16 Feb 2023
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 4 6 0 12 May 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 12 37 0 12 Mar 2022
Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach Luca Furieri C. Galimberti M. Zakwan Giancarlo Ferrari-Trecate 9 20 0 16 Dec 2021
Stable Long-Term Recurrent Video Super-Resolution Benjamin Naoto Chiche Arnaud Woiselle J. Frontera-Pons Jean-Luc Starck SupR 8 5 0 16 Dec 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 38 34 0 23 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks K. Aoyama H. Sawada 11 1 0 01 Sep 2021
Recurrent Quantum Neural Networks Johannes Bausch 21 148 0 25 Jun 2020
HUBERT Untangles BERT to Improve Transfer across NLP Tasks M. Moradshahi Hamid Palangi M. Lam P. Smolensky Jianfeng Gao 13 16 0 25 Oct 2019
Adaptively Truncating Backpropagation Through Time to Control Gradient Bias Christopher Aicher N. Foti E. Fox MQ 14 32 0 17 May 2019
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies A. Chandar Chinnadhurai Sankar Eugene Vorontsov Samira Ebrahimi Kahou Yoshua Bengio 16 56 0 22 Jan 2019
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Q. Ye 28 127 0 29 Jul 2017
Learning Unitary Operators with Help From u(n) Stephanie L. Hyland Gunnar Rätsch 83 42 0 17 Jul 2016