Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1708.07827
Cited By
v1
v2 (latest)
Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study
25 August 2017
Peng Xu
Farbod Roosta-Khorasani
Michael W. Mahoney
ODL
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study"
50 / 69 papers shown
SUPN: Shallow Universal Polynomial Networks
Zachary Morrow
Michael Penwarden
Brian Chen
Aurya Javeed
A. Narayan
J. Jakeman
161
0
0
26 Nov 2025
FUSE: First-Order and Second-Order Unified SynthEsis in Stochastic Optimization
Conference on Algebraic Informatics (AI), 2025
Zhanhong Jiang
Md Zahid Hasan
Aditya Balu
Joshua R. Waite
Genyi Huang
Soumik Sarkar
215
0
0
06 Mar 2025
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
Dahun Shin
Dongyeop Lee
Jinseok Chung
Namhoon Lee
ODL
AAML
1.3K
2
0
25 Feb 2025
Online Control-Informed Learning
Zihao Liang
Tianyu Zhou
Zehui Lu
Shaoshuai Mou
363
5
0
04 Oct 2024
Incremental Gauss-Newton Descent for Machine Learning
Mikalai Korbit
Mario Zanon
ODL
160
0
0
10 Aug 2024
Exact Gauss-Newton Optimization for Training Deep Neural Networks
Mikalai Korbit
Adeyemi Damilare Adeoye
Alberto Bemporad
Mario Zanon
ODL
282
6
0
23 May 2024
Machine Unlearning: A Comprehensive Survey
Weiqi Wang
Zhiyi Tian
Chenhan Zhang
Shui Yu
MU
AILaw
244
33
0
13 May 2024
Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2023
Liu Liu
Xuanqing Liu
Cho-Jui Hsieh
Dacheng Tao
150
5
0
18 Oct 2023
Newton Method-based Subspace Support Vector Data Description
IEEE Symposium Series on Computational Intelligence (IEEE-SSCI), 2023
Fahad Sohrab
Firas Laakom
Moncef Gabbouj
121
6
0
25 Sep 2023
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
Hao-Jun Michael Shi
Tsung-Hsien Lee
Shintaro Iwasaki
Jose Gallego-Posada
Zhijing Li
Kaushik Rangadurai
Dheevatsa Mudigere
Michael Rabbat
ODL
244
43
0
12 Sep 2023
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks
Neural Information Processing Systems (NeurIPS), 2023
Dan Zhao
311
7
0
01 Jun 2023
PyPop7: A Pure-Python Library for Population-Based Black-Box Optimization
Qiqi Duan
Guochen Zhou
Chang Shao
Zhuowei Wang
Mingyang Feng
Yuwei Huang
Yajing Tan
Yijun Yang
Qi Zhao
Yuhui Shi
338
7
0
12 Dec 2022
Adaptive Second Order Coresets for Data-efficient Machine Learning
International Conference on Machine Learning (ICML), 2022
Omead Brandon Pooladzandi
David Davini
Baharan Mirzasoleiman
252
81
0
28 Jul 2022
LogGENE: A smooth alternative to check loss for Deep Healthcare Inference Tasks
Aryaman Jeendgar
Aditya Pola
S. Dhavala
Snehanshu Saha
UQCV
156
3
0
19 Jun 2022
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
Neural Information Processing Systems (NeurIPS), 2022
Saeed Masiha
Saber Salehkaleybar
Niao He
Negar Kiyavash
Patrick Thiran
282
20
0
25 May 2022
HessianFR: An Efficient Hessian-based Follow-the-Ridge Algorithm for Minimax Optimization
Yihang Gao
Huafeng Liu
Michael K. Ng
Mingjie Zhou
150
3
0
23 May 2022
On the efficiency of Stochastic Quasi-Newton Methods for Deep Learning
M. Yousefi
Angeles Martinez
ODL
128
1
0
18 May 2022
Hessian Averaging in Stochastic Newton Methods Achieves Superlinear Convergence
Mathematical programming (Math. Program.), 2022
Sen Na
Michal Derezinski
Michael W. Mahoney
316
21
0
20 Apr 2022
The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
IEEE Conference on Computer Communications (INFOCOM), 2022
Yi Liu
Lei Xu
Lizhen Qu
Cong Wang
Bo Li
MU
144
210
0
14 Mar 2022
Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems
SIAM Journal on Scientific Computing (SISC), 2022
L. Nurbekyan
Wanzhou Lei
Yunbo Yang
214
14
0
13 Feb 2022
Distributed Learning With Sparsified Gradient Differences
IEEE Journal on Selected Topics in Signal Processing (IEEE JSTSP), 2022
Yicheng Chen
Rick S. Blum
Martin Takáč
Brian M. Sadler
221
16
0
05 Feb 2022
Large-Scale Deep Learning Optimizations: A Comprehensive Survey
Xiaoxin He
Fuzhao Xue
Xiaozhe Ren
Yang You
321
18
0
01 Nov 2021
slimTrain -- A Stochastic Approximation Method for Training Separable Deep Neural Networks
Elizabeth Newman
Julianne Chung
Matthias Chung
Lars Ruthotto
206
8
0
28 Sep 2021
Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information
Majid Jahani
S. Rusakov
Zheng Shi
Peter Richtárik
Michael W. Mahoney
Martin Takávc
ODL
178
29
0
11 Sep 2021
Globally Convergent Multilevel Training of Deep Residual Networks
Alena Kopanicáková
Rolf Krause
321
18
0
15 Jul 2021
Nonlinear Least Squares for Large-Scale Machine Learning using Stochastic Jacobian Estimates
Johannes J Brust
157
2
0
12 Jul 2021
Non-PSD Matrix Sketching with Applications to Regression and Optimization
Zhili Feng
Fred Roosta
David P. Woodruff
157
2
0
16 Jun 2021
SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models
International Conference on Learning Representations (ICLR), 2021
Zaccharie Ramzi
Florian Mannel
Shaojie Bai
Jean-Luc Starck
P. Ciuciu
Thomas Moreau
281
33
0
01 Jun 2021
Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionality
International Conference on Machine Learning (ICML), 2021
Jonathan Lacotte
Yifei Wang
Mert Pilanci
188
18
0
15 May 2021
Scalable End-to-End RF Classification: A Case Study on Undersized Dataset Regularization by Convolutional-MST
K. Youssef
Greg Schuette
Yubin Cai
Daisong Zhang
Yikun Huang
Y. Rahmat-Samii
Louis-S. Bouchard
76
1
0
25 Apr 2021
A Distributed Optimisation Framework Combining Natural Gradient with Hessian-Free for Discriminative Sequence Training
Neural Networks (NN), 2021
Adnan Haider
Chao Zhang
Florian Kreyssig
P. Woodland
176
8
0
12 Mar 2021
Better SGD using Second-order Momentum
Neural Information Processing Systems (NeurIPS), 2021
Hoang Tran
Ashok Cutkosky
ODL
184
15
0
04 Mar 2021
Hessian Eigenspectra of More Realistic Nonlinear Models
Neural Information Processing Systems (NeurIPS), 2021
Zhenyu Liao
Michael W. Mahoney
321
38
0
02 Mar 2021
Learning-Augmented Sketches for Hessians
Yi Li
Honghao Lin
David P. Woodruff
180
1
0
24 Feb 2021
Recent Theoretical Advances in Non-Convex Optimization
Marina Danilova
Pavel Dvurechensky
Alexander Gasnikov
Eduard A. Gorbunov
Sergey Guminov
Dmitry Kamzolov
Innokentiy Shibaev
346
104
0
11 Dec 2020
Stochastic Approximation for High-frequency Observations in Data Assimilation
Shushu Zhang
V. Patel
121
1
0
05 Nov 2020
Two-Level K-FAC Preconditioning for Deep Learning
N. Tselepidis
Jonas Köhler
Antonio Orvieto
ODL
140
3
0
01 Nov 2020
BEAR: Sketching BFGS Algorithm for Ultra-High Dimensional Feature Selection in Sublinear Memory
Mathematical and Scientific Machine Learning (MSML), 2020
Amirali Aghazadeh
Vipul Gupta
Alex DeWeese
O. O. Koyluoglu
Kannan Ramchandran
146
2
0
26 Oct 2020
Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra
Vardan Papyan
217
84
0
27 Aug 2020
Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection
SIAM Journal on Mathematics of Data Science (SIMODS), 2020
Elizabeth Newman
Lars Ruthotto
Joseph L. Hart
B. V. B. Waanders
AAML
239
24
0
26 Jul 2020
DynNet: Physics-based neural architecture design for linear and nonlinear structural response modeling and prediction
S. S. Eshkevari
Martin Takávc
S. Pakzad
Majid Jahani
AI4CE
76
12
0
03 Jul 2020
GO Hessian for Expectation-Based Objectives
Yulai Cong
Miaoyun Zhao
Jianqiao Li
Junya Chen
Lawrence Carin
98
0
0
16 Jun 2020
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm
Majid Jahani
M. Nazari
R. Tappenden
A. Berahas
Martin Takávc
ODL
129
10
0
06 Jun 2020
Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations
International Conference on Machine Learning and Applications (ICMLA), 2020
Zheng Shi
Nur Sila Gulgec
A. Berahas
S. Pakzad
Martin Takáč
152
11
0
02 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
AAAI Conference on Artificial Intelligence (AAAI), 2020
Z. Yao
A. Gholami
Sheng Shen
Mustafa Mustafa
Kurt Keutzer
Michael W. Mahoney
ODL
440
333
0
01 Jun 2020
Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
Neural Information Processing Systems (NeurIPS), 2019
Wanxin Jin
Zhaoran Wang
Zhuoran Yang
Shaoshuai Mou
456
110
0
30 Dec 2019
PyHessian: Neural Networks Through the Lens of the Hessian
Z. Yao
A. Gholami
Kurt Keutzer
Michael W. Mahoney
ODL
348
338
0
16 Dec 2019
Quasi-Newton Optimization Methods For Deep Learning Applications
Advances in Intelligent Systems and Computing (AISC), 2019
J. Rafati
Roummel F. Marcia
ODL
124
15
0
04 Sep 2019
Scaling Up Quasi-Newton Algorithms: Communication Efficient Distributed SR1
International Conference on Machine Learning, Optimization, and Data Science (MOD), 2019
Majid Jahani
M. Nazari
S. Rusakov
A. Berahas
Martin Takávc
234
16
0
30 May 2019
An Inertial Newton Algorithm for Deep Learning
Journal of machine learning research (JMLR), 2019
Camille Castera
Jérôme Bolte
Cédric Févotte
Edouard Pauwels
PINN
ODL
262
69
0
29 May 2019
1
2
Next
