Riemannian metrics for neural networks I: feedforward networks

4 March 2013

Papers citing "Riemannian metrics for neural networks I: feedforward networks"

50 / 68 papers shown

Title
Deterministic Bounds and Random Estimates of Metric Tensors on Neuromanifolds Ke Sun 23 0 0 19 May 2025
ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning Nilo Schwencke Cyril Furtlehner 162 1 0 14 Dec 2024
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling Minhyuk Seo Hyunseo Koh Jonghyun Choi 100 3 0 19 Oct 2024
Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models Ji Liu Jiaxiang Ren Ruoming Jin Zijie Zhang Yang Zhou P. Valduriez Dejing Dou FedML 91 6 0 30 Sep 2024
Tradeoffs of Diagonal Fisher Information Matrix Estimators Alexander Soen Ke Sun 71 3 0 08 Feb 2024
An extended asymmetric sigmoid with Perceptron (SIGTRON) for imbalanced linear classification Hyenkyun Woo 42 0 0 26 Dec 2023
Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives Pierre Wolinski ODL 161 0 0 06 Dec 2023
Neural Harmonium: An Interpretable Deep Structure for Nonlinear Dynamic System Identification with Application to Audio Processing Karim Helwani Erfan Soltanmohammadi Michael M. Goodwin 48 0 0 10 Oct 2023
On the Fisher-Rao Gradient of the Evidence Lower Bound Nihat Ay Jesse van Oostrum 15 1 0 20 Jul 2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch Kazuki Osawa Satoki Ishikawa Rio Yokota Shigang Li Torsten Hoefler ODL 92 15 0 08 May 2023
ISAAC Newton: Input-based Approximate Curvature for Newton's Method Felix Petersen Tobias Sutter Christian Borgelt Dongsung Huh Hilde Kuehne Yuekai Sun Oliver Deussen ODL 84 5 0 01 May 2023
Analysis and Comparison of Two-Level KFAC Methods for Training Deep Neural Networks Abdoulaye Koroko A. Anciaux-Sedrakian I. B. Gharbia Valérie Garès M. Haddou Quang-Huy Tran 60 0 0 31 Mar 2023
On a continuous time model of gradient descent dynamics and instability in deep learning Mihaela Rosca Yan Wu Chongli Qin Benoit Dherin 79 10 0 03 Feb 2023
On the Interpretability of Attention Networks L. N. Pandey Rahul Vashisht H. G. Ramaswamy 73 5 0 30 Dec 2022
Adaptive scaling of the learning rate by second order automatic differentiation F. Gournay Alban Gossard ODL 48 2 0 26 Oct 2022
Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 62 1 0 11 Oct 2022
Invariance Properties of the Natural Gradient in Overparametrised Systems Jesse van Oostrum J. Müller Nihat Ay 63 9 0 30 Jun 2022
Information Geometry of Dropout Training Masanari Kimura H. Hino 44 2 0 22 Jun 2022
A Geometric Understanding of Natural Gradient Qinxun Bai S. Rosenberg Wei Xu 70 2 0 13 Feb 2022
A Mini-Block Fisher Method for Deep Neural Networks Achraf Bahamou Shiqian Ma Yi Ren ODL 80 9 0 08 Feb 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization Frederik Benzing ODL 127 26 0 28 Jan 2022
Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition Abdoulaye Koroko A. Anciaux-Sedrakian I. B. Gharbia Valérie Garès M. Haddou Quang-Huy Tran 81 7 0 25 Jan 2022
Gradient representations in ReLU networks as similarity functions Dániel Rácz Balint Daroczy FAtt 52 1 0 26 Oct 2021
On the Variance of the Fisher Information for Deep Learning Alexander Soen Ke Sun FedML FAtt 71 18 0 09 Jul 2021
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka D. Kunin 113 31 0 06 May 2021
Wasserstein Proximal of GANs A. Lin Wuchen Li Stanley Osher Guido Montúfar GAN 50 47 0 13 Feb 2021
Input Similarity from the Neural Network Perspective Guillaume Charpiat N. Girard Loris Felardos Y. Tarabalka 95 76 0 10 Feb 2021
AsymptoticNG: A regularized natural gradient optimization algorithm with look-ahead strategy Zedong Tang Fenlong Jiang Junke Song Maoguo Gong Hao Li F. Yu Zidong Wang Min Wang ODL 27 1 0 24 Dec 2020
Natural-gradient learning for spiking neurons Elena Kreutzer Walter Senn Mihai A. Petrovici 38 13 0 23 Nov 2020
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks Ryo Karakida Kazuki Osawa 80 26 0 02 Oct 2020
Natural Reweighted Wake-Sleep Csongor-Huba Várady Riccardo Volpi Luigi Malagò Nihat Ay BDL 99 0 0 15 Aug 2020
On the Locality of the Natural Gradient for Deep Learning Nihat Ay 11 0 0 21 May 2020
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent Pu Zhao Pin-Yu Chen Siyue Wang Xinyu Lin AAML 73 36 0 18 Feb 2020
Tangent Space Separability in Feedforward Neural Networks Balint Daroczy Rita Aleksziev András A. Benczúr 45 3 0 18 Dec 2019
Neural Spectrum Alignment: Empirical Study Dmitry Kopitkov Vadim Indelman 88 14 0 19 Oct 2019
Pathological spectra of the Fisher information metric and its variants in deep neural networks Ryo Karakida S. Akaho S. Amari 65 28 0 14 Oct 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 101 219 0 29 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 113 126 0 27 May 2019
The Extended Kalman Filter is a Natural Gradient Descent in Trajectory Space Yann Ollivier 82 21 0 03 Jan 2019
First-order and second-order variants of the gradient descent in a unified framework Thomas Pierrot Nicolas Perrin Olivier Sigaud ODL 69 7 0 18 Oct 2018
A Coordinate-Free Construction of Scalable Natural Gradient Kevin Luk Roger C. Grosse 56 11 0 30 Aug 2018
Fisher Information and Natural Gradient Learning of Random Deep Networks S. Amari Ryo Karakida Masafumi Oizumi 71 36 0 22 Aug 2018
Expressive power of outer product manifolds on feed-forward neural networks Balint Daroczy Rita Aleksziev András A. Benczúr 29 0 0 17 Jul 2018
Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis Thomas George César Laurent Xavier Bouthillier Nicolas Ballas Pascal Vincent ODL 103 156 0 11 Jun 2018
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 193 146 0 04 Jun 2018
Sigsoftmax: Reanalysis of the Softmax Bottleneck Sekitoshi Kanai Yasuhiro Fujiwara Yuki Yamanaka S. Adachi 74 70 0 28 May 2018
Block Mean Approximation for Efficient Second Order Optimization Yao Lu Mehrtash Harandi Leonid Sigal Razvan Pascanu ODL 62 4 0 16 Apr 2018
Accelerating Natural Gradient with Higher-Order Invariance Yang Song Jiaming Song Stefano Ermon 72 23 0 04 Mar 2018
Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation Huishuai Zhang Wei-neng Chen Tie-Yan Liu 27 5 0 27 Feb 2018
Degeneration in VAE: in the Light of Fisher Information Loss Huangjie Zheng Jiangchao Yao Ya Zhang Ivor W. Tsang DRL 78 17 0 19 Feb 2018