Analytic Insights into Structure and Rank of Neural Network Hessian Maps

Analytic Insights into Structure and Rank of Neural Network Hessian Maps

30 June 2021

Sidak Pal Singh

Gregor Bachmann

Papers citing "Analytic Insights into Structure and Rank of Neural Network Hessian Maps"

11 / 11 papers shown

Title
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 58 3 0 31 Jan 2025
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 45 0 0 04 Nov 2024
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec Felix Dangel Sidak Pal Singh 33 6 0 14 Oct 2024
Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes Nikita Kiselev Andrey Grabovoy 51 1 0 18 Sep 2024
Sampling Foundational Transformer: A Theoretical Perspective Viet Anh Nguyen Minh Lenhat Khoa Nguyen Duong Duc Hieu Dao Huu Hung Truong Son-Hy 44 0 0 11 Aug 2024
On Newton's Method to Unlearn Neural Networks Nhung Bui Xinyang Lu Rachael Hwee Ling Sim See-Kiong Ng Bryan Kian Hsiang Low MU 41 2 0 20 Jun 2024
Robust low-rank training via approximate orthonormal constraints Dayana Savostianova Emanuele Zangrando Gianluca Ceruti Francesco Tudisco 24 9 0 02 Jun 2023
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 28 6 0 05 Dec 2022
Federated Optimization of Smooth Loss Functions Ali Jadbabaie A. Makur Devavrat Shah FedML 21 7 0 06 Jan 2022
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 281 2,889 0 15 Sep 2016
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 119 577 0 27 Feb 2015