Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations

24 June 2020

Papers citing "Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations"

32 / 32 papers shown

Title
FUSE: First-Order and Second-Order Unified SynthEsis in Stochastic Optimization Zhanhong Jiang Md Zahid Hasan Aditya Balu Joshua R. Waite Genyi Huang S. Sarkar 49 0 0 06 Mar 2025
A stochastic first-order method with multi-extrapolated momentum for highly smooth unconstrained optimization Chuan He 75 0 0 19 Dec 2024
General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization Kwangjun Ahn Gagik Magakyan Ashok Cutkosky 56 0 0 11 Nov 2024
Empirical Tests of Optimization Assumptions in Deep Learning Hoang Tran Qinzi Zhang Ashok Cutkosky 31 1 0 01 Jul 2024
Private Zeroth-Order Nonsmooth Nonconvex Optimization Qinzi Zhang Hoang Tran Ashok Cutkosky 38 4 0 27 Jun 2024
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 35 4 0 16 May 2024
Learning Optimal Deterministic Policies with Stochastic Policy Gradients Alessandro Montenegro Marco Mussi Alberto Maria Metelli Matteo Papini 40 2 0 03 May 2024
On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimization Ling Liang Haizhao Yang 14 0 0 23 Jan 2024
A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees Shuyao Li Stephen J. Wright 16 3 0 28 Oct 2023
A Homogenization Approach for Gradient-Dominated Stochastic Optimization Jiyuan Tan Chenyu Xue Chuwen Zhang Qi Deng Dongdong Ge Yinyu Ye 23 2 0 21 Aug 2023
Debiasing Conditional Stochastic Optimization Lie He S. Kasiviswanathan CML BDL 33 4 0 20 Apr 2023
Deterministic Nonsmooth Nonconvex Optimization Michael I. Jordan Guy Kornowski Tianyi Lin Ohad Shamir Manolis Zampetakis 49 24 0 16 Feb 2023
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion Ashok Cutkosky Harsh Mehta Francesco Orabona 33 32 0 07 Feb 2023
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies Ilyas Fatkhullin Anas Barakat Anastasia Kireeva Niao He 19 37 0 03 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 19 1 0 28 Jan 2023
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 24 3 0 12 Dec 2022
Momentum Aggregation for Private Non-convex ERM Hoang Tran Ashok Cutkosky 8 14 0 12 Oct 2022
Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization Tianyi Lin Zeyu Zheng Michael I. Jordan 49 51 0 12 Sep 2022
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models Xingyu Xie Pan Zhou Huan Li Zhouchen Lin Shuicheng Yan ODL 35 147 0 13 Aug 2022
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function Saeed Masiha Saber Salehkaleybar Niao He Negar Kiyavash Patrick Thiran 79 18 0 25 May 2022
Momentum-Based Policy Gradient with Second-Order Information Saber Salehkaleybar Sadegh Khorasani Negar Kiyavash Niao He Patrick Thiran 21 9 0 17 May 2022
Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions Xu Cai Chaobing Song Cristóbal Guzmán Jelena Diakonikolas 32 9 0 17 Mar 2022
On the Second-order Convergence Properties of Random Search Methods Aurélien Lucchi Antonio Orvieto Adamos Solomou 16 8 0 25 Oct 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 25 1 0 25 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 26 15 0 07 Oct 2021
High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails Ashok Cutkosky Harsh Mehta 22 55 0 28 Jun 2021
On the Oracle Complexity of Higher-Order Smooth Non-Convex Finite-Sum Optimization N. Emmenegger Rasmus Kyng Ahad N. Zehmakan 10 2 0 08 Mar 2021
Better SGD using Second-order Momentum Hoang Tran Ashok Cutkosky ODL 9 12 0 04 Mar 2021
Stochastic Approximation for Online Tensorial Independent Component Analysis C. J. Li Michael I. Jordan 25 2 0 28 Dec 2020
Recent Theoretical Advances in Non-Convex Optimization Marina Danilova Pavel Dvurechensky Alexander Gasnikov Eduard A. Gorbunov Sergey Guminov Dmitry Kamzolov Innokentiy Shibaev 18 76 0 11 Dec 2020
Lower Bounds for Non-Convex Stochastic Optimization Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Nathan Srebro Blake E. Woodworth 15 340 0 05 Dec 2019
A Sub-sampled Tensor Method for Non-convex Optimization Aurélien Lucchi Jonas Köhler 17 0 0 23 Nov 2019