New insights and perspectives on the natural gradient method

3 December 2014

Papers citing "New insights and perspectives on the natural gradient method"

50 / 125 papers shown

Title
Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 15 1 0 11 Oct 2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization Gábor Melis MoMe 36 1 0 26 Sep 2022
FedFOR: Stateless Heterogeneous Federated Learning with First-Order Regularization Junjiao Tian James Smith Z. Kira 19 3 0 21 Sep 2022
Conjugate Natural Selection Reilly P. Raab Luca de Alfaro Yang Liu 18 0 0 29 Aug 2022
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices Mingbin Xu Congzheng Song Ye Tian Neha Agrawal Filip Granqvist ... Shiyi Han Yaqiao Deng Leo Liu Anmol Walia Alex Jin FedML 15 22 0 18 Jul 2022
Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning Lin Zhang S. Shi Wei Wang Bo-wen Li 36 10 0 30 Jun 2022
Classification of datasets with imputed missing values: does imputation quality matter? Tolou Shadbahr M. Roberts Jan Stanczuk J. Gilbey P. Teare ... T. Mirtti A. Rannikko J. Aston Jing Tang Carola-Bibiane Schönlieb 30 52 0 16 Jun 2022
Attack-Agnostic Adversarial Detection Jiaxin Cheng Mohamed Hussein J. Billa Wael AbdAlmageed AAML 26 0 0 01 Jun 2022
DIRA: Dynamic Domain Incremental Regularised Adaptation Abanoub Ghobrial Xu Zheng Darryl Hond Hamid Asgari Kerstin Eder AI4CE 26 1 0 30 Apr 2022
Rethinking Exponential Averaging of the Fisher C. Puiu 23 1 0 10 Apr 2022
Bayesian Learning Approach to Model Predictive Control Namhoon Cho Seokwon Lee Hyo-Sang Shin Antonios Tsourdos 18 1 0 05 Mar 2022
Model-agnostic out-of-distribution detection using combined statistical tests Federico Bergamin Pierre-Alexandre Mattei Jakob Drachmann Havtorn Hugo Senetaire Hugo Schmutz Lars Maaløe Søren Hauberg J. Frellsen OODD 24 18 0 02 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 27 14 0 28 Feb 2022
Continual learning-based probabilistic slow feature analysis for multimode dynamic process monitoring Jingxin Zhang Donghua Zhou Maoyin Chen Xia Hong 21 14 0 23 Feb 2022
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations Alexander Immer Tycho F. A. van der Ouderaa Gunnar Rätsch Vincent Fortuin Mark van der Wilk BDL 39 44 0 22 Feb 2022
Classical versus Quantum: comparing Tensor Network-based Quantum Circuits on LHC data Jack Y. Araz M. Spannowsky 36 14 0 21 Feb 2022
A Geometric Understanding of Natural Gradient Qinxun Bai S. Rosenberg Wei Xu 21 2 0 13 Feb 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization Frederik Benzing ODL 43 23 0 28 Jan 2022
Dynamically Stable Poincaré Embeddings for Neural Manifolds Jun Chen Yuang Liu Xiangrui Zhao Mengmeng Wang Yong-Jin Liu 26 0 0 21 Dec 2021
Depth Without the Magic: Inductive Bias of Natural Gradient Descent A. Kerekes Anna Mészáros Ferenc Huszár ODL 29 4 0 22 Nov 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 32 197 0 18 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 30 14 0 01 Nov 2021
Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis Yi Liu Yuanshao Zhu James J. Q. Yu FedML 27 28 0 14 Oct 2021
The Information Geometry of Unsupervised Reinforcement Learning Benjamin Eysenbach Ruslan Salakhutdinov Sergey Levine SSL OffRL 61 31 0 06 Oct 2021
Approximate Newton policy gradient algorithms Haoya Li Samarth Gupta Hsiangfu Yu Lexing Ying Inderjit Dhillon 51 2 0 05 Oct 2021
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning Prasetya Ajie Utama N. Moosavi Victor Sanh Iryna Gurevych AAML 61 35 0 09 Sep 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 40 204 0 07 Sep 2021
The Bayesian Learning Rule Mohammad Emtiyaz Khan Håvard Rue BDL 63 73 0 09 Jul 2021
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information Elias Frantar Eldar Kurtic Dan Alistarh 13 57 0 07 Jul 2021
Certifiable Machine Unlearning for Linear Models Ananth Mahadevan M. Mathioudakis MU 14 45 0 29 Jun 2021
Laplace Redux -- Effortless Bayesian Deep Learning Erik A. Daxberger Agustinus Kristiadi Alexander Immer Runa Eschenhagen Matthias Bauer Philipp Hennig BDL UQCV 58 289 0 28 Jun 2021
Sampling with Mirrored Stein Operators Jiaxin Shi Chang-rui Liu Lester W. Mackey 45 19 0 23 Jun 2021
Natural continual learning: success is a journey, not (just) a destination Ta-Chu Kao Kristopher T. Jensen Gido M. van de Ven A. Bernacchia Guillaume Hennequin CLL 23 46 0 15 Jun 2021
Pulling back information geometry Georgios Arvanitidis Miguel González Duque Alison Pouplin Dimitris Kalatzis Søren Hauberg DRL 16 14 0 09 Jun 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 24 10 0 07 Jun 2021
Local Adaptivity in Federated Learning: Convergence and Consistency Jianyu Wang Zheng Xu Zachary Garrett Zachary B. Charles Luyang Liu Gauri Joshi FedML 32 39 0 04 Jun 2021
Reweighting Augmented Samples by Minimizing the Maximal Expected Loss Mingyang Yi Lu Hou Lifeng Shang Xin Jiang Qun Liu Zhi-Ming Ma 12 19 0 16 Mar 2021
A practical tutorial on Variational Bayes Minh-Ngoc Tran Trong-Nghia Nguyen Viet-Hung Dao BDL 29 38 0 01 Mar 2021
Posterior Meta-Replay for Continual Learning Christian Henning Maria R. Cervera Francesco DÁngelo J. Oswald Regina Traber Benjamin Ehret Seijin Kobayashi Benjamin Grewe João Sacramento CLL BDL 51 54 0 01 Mar 2021
Appearance of Random Matrix Theory in Deep Learning Nicholas P. Baskerville Diego Granziol J. Keating 15 11 0 12 Feb 2021
Mirror-Descent Inverse Kinematics for Box-constrained Joint Space Taisuke Kobayashi Takanori Jin 46 2 0 19 Jan 2021
Generalized Variational Continual Learning Noel Loo S. Swaroop Richard Turner BDL CLL 33 58 0 24 Nov 2020
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient Kai-Xin Gao Xiaolei Liu Zheng-Hai Huang Min Wang Zidong Wang Dachuan Xu F. Yu 24 11 0 21 Nov 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 29 2 0 15 Nov 2020
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 44 6 0 09 Nov 2020
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians Juhan Bae Roger C. Grosse 27 24 0 26 Oct 2020
VacSIM: Learning Effective Strategies for COVID-19 Vaccine Distribution using Reinforcement Learning R. Awasthi K. K. Guliani Saif Ahmad Khan Aniket Vashishtha M. S. Gill Arshita Bhatt A. Nagori Aniket Gupta Ponnurangam Kumaraguru Tavpritesh Sethi 34 24 0 14 Sep 2020
Optimization of Graph Neural Networks with Natural Gradient Descent M. Izadi Yihao Fang R. Stevenson Lizhen Lin GNN 24 41 0 21 Aug 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 30 2 0 15 Aug 2020
Natural Gradient Shared Control Yoojin Oh Shaowen Wu Marc Toussaint Jim Mainprice 13 9 0 30 Jul 2020