Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for
Regression Problems

Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems

28 May 2019

Tianle Cai

Ruiqi Gao

Papers citing "Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems"

18 / 18 papers shown

Title
ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning Nilo Schwencke Cyril Furtlehner 64 1 0 14 Dec 2024
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao-quan Song Chiwun Yang 34 29 0 23 Aug 2023
$A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee$ A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee Zhao-quan Song Mingquan Ye Junze Yin Licheng Zhang 14 10 0 01 Feb 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 71 31 0 25 Nov 2022
Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 11 1 0 11 Oct 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao-quan Song Licheng Zhang Ruizhe Zhang 23 63 0 14 Dec 2021
SCORE: Approximating Curvature Information under Self-Concordant Regularization Adeyemi Damilare Adeoye Alberto Bemporad 13 4 0 14 Dec 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 34 6 0 15 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao-quan Song Shuo Yang Ruizhe Zhang 32 49 0 09 Oct 2021
Improved architectures and training algorithms for deep operator networks Sifan Wang Hanwen Wang P. Perdikaris AI4CE 47 105 0 04 Oct 2021
Credit Assignment in Neural Networks through Deep Feedback Control Alexander Meulemans Matilde Tristany Farinha Javier García Ordónez Pau Vilimelis Aceituno João Sacramento Benjamin Grewe 23 34 0 15 Jun 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 21 10 0 07 Jun 2021
Appearance of Random Matrix Theory in Deep Learning Nicholas P. Baskerville Diego Granziol J. Keating 13 11 0 12 Feb 2021
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 16 158 0 07 Sep 2020
Scalable Derivative-Free Optimization for Nonlinear Least-Squares Problems C. Cartis T. Ferguson Lindon Roberts 21 6 0 26 Jul 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 34 32 0 18 Jun 2020
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective S. Amari 13 12 0 20 Jan 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 281 2,888 0 15 Sep 2016