New insights and perspectives on the natural gradient method

3 December 2014

Papers citing "New insights and perspectives on the natural gradient method"

25 / 125 papers shown

Title
Adversarial Training Reduces Information and Improves Transferability M. Terzi Alessandro Achille Marco Maggipinto Gian Antonio Susto AAML 24 23 0 22 Jul 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 37 49 0 16 Jun 2020
Elastic weight consolidation for better bias inoculation James Thorne Andreas Vlachos 22 11 0 29 Apr 2020
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting Sanyuan Chen Yutai Hou Yiming Cui Wanxiang Che Ting Liu Xiangzhan Yu KELM CLL 21 212 0 27 Apr 2020
RelatIF: Identifying Explanatory Training Examples via Relative Influence Elnaz Barshan Marc-Etienne Brunet Gintare Karolina Dziugaite TDI 47 30 0 25 Mar 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 10 3 0 02 Mar 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 25 168 0 19 Dec 2019
Hierarchical model-based policy optimization: from actions to action sequences and back Daniel C. McNamee 11 1 0 28 Nov 2019
Geometry of learning neural quantum states Chae-Yeun Park M. Kastoryano 24 60 0 24 Oct 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 48 719 0 19 Jul 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 21 207 0 29 May 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 20 22 0 21 Feb 2019
Trust Region Value Optimization using Kalman Filtering Shirli Di-Castro Shashua Shie Mannor 19 7 0 23 Jan 2019
First-order and second-order variants of the gradient descent in a unified framework Thomas Pierrot Nicolas Perrin Olivier Sigaud ODL 30 7 0 18 Oct 2018
Fisher Information and Natural Gradient Learning of Random Deep Networks S. Amari Ryo Karakida Masafumi Oizumi 19 34 0 22 Aug 2018
On the Acceleration of L-BFGS with Second-Order Information and Stochastic Batches Jie Liu Yu Rong Martin Takáč Junzhou Huang ODL 24 7 0 14 Jul 2018
Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers Marcel Hirt P. Dellaportas BDL 20 10 0 23 May 2018
Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training Boyu Chen Wenlian Lu Ernest Fokoue 21 1 0 22 May 2018
Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models Hugh Salimbeni Stefanos Eleftheriadis J. Hensman BDL 20 85 0 24 Mar 2018
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes Erin Grant Chelsea Finn Sergey Levine Trevor Darrell Thomas L. Griffiths BDL 21 505 0 26 Jan 2018
Warped Riemannian metrics for location-scale models Salem Said Lionel Bombrun Y. Berthoumieu 35 15 0 22 Jul 2017
YellowFin and the Art of Momentum Tuning Jian Zhang Ioannis Mitliagkas ODL 23 108 0 12 Jun 2017
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 22 3,172 0 15 Jun 2016