Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives

v1v2v3 (latest)

Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives

6 December 2023

Pierre Wolinski

ArXiv (abs)PDF HTML

Papers citing "Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives"

11 / 11 papers shown

Title
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 205 1,360 0 03 Oct 2020
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 161 1,439 0 01 May 2019
Are All Layers Created Equal? Chiyuan Zhang Samy Bengio Y. Singer 111 140 0 06 Feb 2019
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 346 3,226 0 20 Jun 2018
Block Mean Approximation for Efficient Second Order Optimization Yao Lu Mehrtash Harandi Leonid Sigal Razvan Pascanu ODL 51 4 0 16 Apr 2018
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks Levent Sagun Utku Evci V. U. Güney Yann N. Dauphin Léon Bottou 98 420 0 14 Jun 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 147 774 0 15 Mar 2017
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) Djork-Arné Clevert Thomas Unterthiner Sepp Hochreiter 321 5,543 0 23 Nov 2015
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 141 1,025 0 19 Mar 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.8K 100,712 0 04 Sep 2014
Riemannian metrics for neural networks I: feedforward networks Yann Ollivier 117 104 0 04 Mar 2013