Scalable Second Order Optimization for Deep Learning

Scalable Second Order Optimization for Deep Learning

20 February 2020

Papers citing "Scalable Second Order Optimization for Deep Learning"

19 / 19 papers shown

Title
PETScML: Second-order solvers for training regression problems in Scientific Machine Learning Stefano Zampini Umberto Zerbinati George Turkyyiah David E. Keyes 35 4 0 18 Mar 2024
Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics Hanlin Yu M. Hartmann Bernardo Williams Arto Klami BDL 14 4 0 09 Mar 2023
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 21 60 0 17 Nov 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 29 47 0 13 Oct 2022
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models Rohan Anil S. Gadanho Danya Huang Nijith Jacob Zhuoshu Li ... Cristina Pop Kevin Regan G. Shamir Rakesh Shivanna Qiqi Yan 3DV 10 41 0 12 Sep 2022
Hessian-Free Second-Order Adversarial Examples for Adversarial Learning Yaguan Qian Yu-qun Wang Bin Wang Zhaoquan Gu Yu-Shuang Guo Wassim Swaileh AAML 39 3 0 04 Jul 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 33 32 0 22 Mar 2022
Local Quadratic Convergence of Stochastic Gradient Descent with Adaptive Step Size Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler ODL 15 0 0 30 Dec 2021
Real-time Neural Radiance Caching for Path Tracing Thomas Müller Fabrice Rousselle Jan Novák A. Keller 3DH AI4CE 16 155 0 23 Jun 2021
A Generalizable Approach to Learning Optimizers Diogo Almeida Clemens Winter Jie Tang Wojciech Zaremba AI4CE 16 29 0 02 Jun 2021
AsymptoticNG: A regularized natural gradient optimization algorithm with look-ahead strategy Zedong Tang Fenlong Jiang Junke Song Maoguo Gong Hao Li F. Yu Zidong Wang Min Wang ODL 15 1 0 24 Dec 2020
Second-order Neural Network Training Using Complex-step Directional Derivative Siyuan Shen Tianjia Shao Kun Zhou Chenfanfu Jiang Feng Luo Yin Yang ODL 11 2 0 15 Sep 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 32 161 0 03 Jul 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao-quan Song Omri Weinstein ODL 13 81 0 20 Jun 2020
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training Hongyu Zhu Amar Phanishayee Gennady Pekhimenko 13 52 0 05 Jun 2020
PyHessian: Neural Networks Through the Lens of the Hessian Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 8 288 0 16 Dec 2019
Zap Q-Learning With Nonlinear Function Approximation Shuhang Chen Adithya M. Devraj Fan Lu Ana Bušić Sean P. Meyn 17 20 0 11 Oct 2019
An Adaptive Remote Stochastic Gradient Method for Training Neural Networks Yushu Chen Hao Jing Wenlai Zhao Zhiqiang Liu H. Fu Lián Qiao Wei Xue Guangwen Yang ODL 19 2 0 04 May 2019
OverSketched Newton: Fast Convex Optimization for Serverless Systems Vipul Gupta S. Kadhe T. Courtade Michael W. Mahoney K. Ramchandran 16 33 0 21 Mar 2019