New insights and perspectives on the natural gradient method

3 December 2014

Papers citing "New insights and perspectives on the natural gradient method"

50 / 121 papers shown

Title
Variational Formulation of the Particle Flow Particle Filter Yinzhuang Yi Jorge Cortés Nikolay Atanasov 36 0 0 06 May 2025
NANO-SLAM : Natural Gradient Gaussian Approximation for Vehicle SLAM Tianyi Zhang Wenhan Cao Chang-rui Liu Feihong Zhang Wei Wu Shengbo Eben Li 24 0 0 27 Apr 2025
Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis Ruijun Deng Zhihui Lu Qiang Duan FedML 46 0 0 14 Apr 2025
Enhancing Multi-task Learning Capability of Medical Generalist Foundation Model via Image-centric Multi-annotation Data Xun Zhu Fanbin Mo Zheng Zhang J. Wang Yiming Shi Ming Wu Chuang Zhang Miao Li Ji Wu 32 0 0 14 Apr 2025
CAMEx: Curvature-aware Merging of Experts Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 104 2 0 26 Feb 2025
Data Attribution for Text-to-Image Models by Unlearning Synthesized Images Sheng-Yu Wang Aaron Hertzmann Alexei A. Efros Jun-Yan Zhu Richard Zhang TDI 128 2 0 21 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 58 3 0 31 Jan 2025
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 48 0 0 04 Nov 2024
Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models Jinxu Lin Linwei Tao Minjing Dong Chang Xu TDI 41 2 0 24 Oct 2024
Debiasing Mini-Batch Quadratics for Applications in Deep Learning Lukas Tatzel Bálint Mucsányi Osane Hackel Philipp Hennig 43 0 0 18 Oct 2024
Influence Functions for Scalable Data Attribution in Diffusion Models Bruno Mlodozeniec Runa Eschenhagen Juhan Bae Alexander Immer David Krueger Richard E. Turner TDI DiffM 75 4 0 17 Oct 2024
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec Felix Dangel Sidak Pal Singh 38 7 0 14 Oct 2024
Erasure Coded Neural Network Inference via Fisher Averaging Divyansh Jhunjhunwala Neharika Jali Gauri Joshi Shiqiang Wang MoMe FedML 31 1 0 02 Sep 2024
An Improved Empirical Fisher Approximation for Natural Gradient Descent Xiaodong Wu Wenyi Yu Chao Zhang Philip Woodland 29 3 0 10 Jun 2024
A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks Hengzhu Liu Ping Xiong Tianqing Zhu Philip S. Yu 35 6 0 10 Jun 2024
Linearization Turns Neural Operators into Function-Valued Gaussian Processes Emilia Magnani Marvin Pfortner Tobias Weber Philipp Hennig UQCV 69 1 0 07 Jun 2024
Scalable Bayesian Learning with posteriors Samuel Duffield Kaelan Donatella Johnathan Chiu Phoebe Klett Daniel Simpson BDL UQCV 62 1 0 31 May 2024
Categorical Flow Matching on Statistical Manifolds Chaoran Cheng Jiahan Li Jian-wei Peng Ge Liu 61 10 0 26 May 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 76 2 0 26 May 2024
Pruning as a Domain-specific LLM Extractor Nan Zhang Yanchi Liu Xujiang Zhao Wei Cheng Runxue Bao Rui Zhang Prasenjit Mitra Haifeng Chen 26 9 0 10 May 2024
WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling Huai-an Su Jiaxiang Geng Liang Li Xiaoqi Qin Yanzhao Hou Xin Fu Miao Pan Miao Pan 40 1 0 01 May 2024
Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications Lucas Böttcher Gregory R. Wheeler 32 0 0 05 Apr 2024
Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations Huan Zhang Yifan Chen Eric Vanden-Eijnden Benjamin Peherstorfer 42 2 0 01 Apr 2024
Discrete Natural Evolution Strategies Ahmad Ayaz Amin 25 0 0 30 Mar 2024
A Differential Geometric View and Explainability of GNN on Evolving Graphs Yazheng Liu Xi Zhang Sihong Xie 21 3 0 11 Mar 2024
Training-set-free two-stage deep learning for spectroscopic data de-noising Dongchen Huang Junde Liu Tian Qian Hongming Weng 36 0 0 29 Feb 2024
Corrective Machine Unlearning Shashwat Goel Ameya Prabhu Philip Torr Ponnurangam Kumaraguru Amartya Sanyal OnRL 40 14 0 21 Feb 2024
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC Wu Lin Felix Dangel Runa Eschenhagen Kirill Neklyudov Agustinus Kristiadi Richard Turner Alireza Makhzani 22 3 0 09 Dec 2023
Intriguing Properties of Data Attribution on Diffusion Models Xiaosen Zheng Tianyu Pang Chao Du Jing Jiang Min-Bin Lin TDI 34 20 1 01 Nov 2023
Model Merging by Uncertainty-Based Gradient Matching Nico Daheim Thomas Möllenhoff E. Ponti Iryna Gurevych Mohammad Emtiyaz Khan MoMe FedML 32 43 0 19 Oct 2023
Modify Training Directions in Function Space to Reduce Generalization Error Yi Yu Wenlian Lu Boyu Chen 27 0 0 25 Jul 2023
Correlated Noise in Epoch-Based Stochastic Gradient Descent: Implications for Weight Variances Marcel Kühn B. Rosenow 19 3 0 08 Jun 2023
KrADagrad: Kronecker Approximation-Domination Gradient Preconditioned Stochastic Optimization Jonathan Mei Alexander Moreno Luke Walters ODL 29 1 0 30 May 2023
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training Hong Liu Zhiyuan Li David Leo Wright Hall Percy Liang Tengyu Ma VLM 55 130 0 23 May 2023
ISAAC Newton: Input-based Approximate Curvature for Newton's Method Felix Petersen Tobias Sutter Christian Borgelt Dongsung Huh Hilde Kuehne Yuekai Sun Oliver Deussen ODL 31 5 0 01 May 2023
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation Nico Daheim Nouha Dziri Mrinmaya Sachan Iryna Gurevych E. Ponti MoMe 34 30 0 30 Mar 2023
A Multi-Agent Adaptive Deep Learning Framework for Online Intrusion Detection Mahdi Soltani Khashayar Khajavi M. J. Siavoshani A. Jahangir 18 7 0 05 Mar 2023
Achieving High Accuracy with PINNs via Energy Natural Gradients Johannes Müller Marius Zeinhofer 13 4 0 25 Feb 2023
Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance Yifan Chen Daniel Zhengyu Huang Jiaoyang Huang Sebastian Reich Andrew M. Stuart 19 17 0 21 Feb 2023
Structural Neural Additive Models: Enhanced Interpretable Machine Learning Mattias Luber Anton Thielmann Benjamin Säfken 31 7 0 18 Feb 2023
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 32 11 0 14 Feb 2023
Learning Discretized Neural Networks under Ricci Flow Jun Chen Han Chen Mengmeng Wang Guang Dai Ivor W. Tsang Yong-Jin Liu 25 2 0 07 Feb 2023
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing Chen Cecilia Liu Jonas Pfeiffer Ivan Vulić Iryna Gurevych CLL 29 9 0 13 Jan 2023
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices Kazuki Osawa Shigang Li Torsten Hoefler AI4CE 35 24 0 25 Nov 2022
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods Yanli Liu Kaipeng Zhang Tamer Basar W. Yin 45 102 0 15 Nov 2022
HesScale: Scalable Computation of Hessian Diagonals Mohamed Elsayed A. R. Mahmood 22 7 0 20 Oct 2022
Brand New K-FACs: Speeding up K-FAC with Online Decomposition Updates C. Puiu 14 2 0 16 Oct 2022
On the Identifiability and Estimation of Causal Location-Scale Noise Models Alexander Immer Christoph Schultheiss Julia E. Vogt Bernhard Schölkopf Peter Buhlmann Alexander Marx CML 31 32 0 13 Oct 2022
Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 15 1 0 11 Oct 2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization Gábor Melis MoMe 36 1 0 26 Sep 2022