Natasha 2: Faster Non-Convex Optimization Than SGD

29 August 2017

Papers citing "Natasha 2: Faster Non-Convex Optimization Than SGD"

49 / 49 papers shown

Title
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning Jingruo Sun Zachary Frangella Madeleine Udell 31 0 0 28 Jan 2025
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 35 4 0 16 May 2024
Private Federated Learning with Dynamic Power Control via Non-Coherent Over-the-Air Computation Anbang Zhang Shuaishuai Guo Shuai Liu 19 2 0 05 Aug 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 29 6 0 25 May 2023
Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization G. Fort Eric Moulines 46 6 0 02 Jan 2023
Adaptive scaling of the learning rate by second order automatic differentiation F. Gournay Alban Gossard ODL 23 1 0 26 Oct 2022
Optimization for Amortized Inverse Problems Tianci Liu Tong Yang Quan Zhang Qi Lei 28 4 0 25 Oct 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 22 7 0 04 Oct 2022
Interference Management for Over-the-Air Federated Learning in Multi-Cell Wireless Networks Zhibin Wang Yong Zhou Yuanming Shi W. Zhuang 25 67 0 06 Jun 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 18 8 0 18 Feb 2022
Coordinate Descent Methods for Fractional Minimization Ganzhao Yuan 13 6 0 30 Jan 2022
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 23 15 0 28 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 25 1 0 25 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 26 15 0 07 Oct 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 20 13 0 19 Jul 2021
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent Spencer Frei Quanquan Gu 17 25 0 25 Jun 2021
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis Stephan Wojtowytsch 23 50 0 04 May 2021
Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer Qunsong Zeng Yuqing Du Kaibin Huang 24 35 0 24 Feb 2021
PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization Zhize Li Hongyan Bao Xiangliang Zhang Peter Richtárik ODL 24 125 0 25 Aug 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 77 53 0 24 Jun 2020
Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent Shuang Song Thomas Steinke Om Thakkar Abhradeep Thakurta 27 50 0 11 Jun 2020
Statistical Learning with Conditional Value at Risk Tasuku Soma Yuichi Yoshida 6 38 0 14 Feb 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Ming-Yu Liu ODL 195 57 0 09 Feb 2020
One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis Guangxu Zhu Yuqing Du Deniz Gunduz Kaibin Huang 31 308 0 16 Jan 2020
Energy Efficient Federated Learning Over Wireless Communication Networks Zhaohui Yang Mingzhe Chen Walid Saad C. Hong M. Shikh-Bahaei 13 680 0 06 Nov 2019
Combining Stochastic Adaptive Cubic Regularization with Negative Curvature for Nonconvex Optimization Seonho Park Seung Hyun Jung P. Pardalos ODL 16 15 0 27 Jun 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 17 33 0 01 May 2019
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization Nhan H. Pham Lam M. Nguyen Dzung Phan Quoc Tran-Dinh 11 139 0 15 Feb 2019
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate J. Zhang Hongyi Zhang S. Sra 16 39 0 10 Nov 2018
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron Sharan Vaswani Francis R. Bach Mark W. Schmidt 28 296 0 16 Oct 2018
signSGD with Majority Vote is Communication Efficient And Fault Tolerant Jeremy Bernstein Jiawei Zhao Kamyar Azizzadenesheli Anima Anandkumar FedML 9 46 0 11 Oct 2018
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property Yi Zhou Zhe Wang Yingbin Liang 21 23 0 22 Aug 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 33 568 0 04 Jul 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen K. Ramchandran Peter L. Bartlett FedML 24 97 0 14 Jun 2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes Rachel A. Ward Xiaoxia Wu Léon Bottou ODL 19 358 0 05 Jun 2018
Stochastic Zeroth-order Optimization via Variance Reduction method L. Liu Minhao Cheng Cho-Jui Hsieh Dacheng Tao 19 19 0 30 May 2018
Local Saddle Point Optimization: A Curvature Exploitation Approach Leonard Adolphs Hadi Daneshmand Aurélien Lucchi Thomas Hofmann 15 107 0 15 May 2018
Stochastic model-based minimization of weakly convex functions Damek Davis D. Drusvyatskiy 17 370 0 17 Mar 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurélien Lucchi Thomas Hofmann 19 161 0 15 Mar 2018
A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 33 116 0 13 Feb 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu-Xiang Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 30 1,018 0 13 Feb 2018
Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization Zeyuan Allen-Zhu ODL 42 52 0 12 Feb 2018
Neon2: Finding Local Minima via First-Order Oracles Zeyuan Allen-Zhu Yuanzhi Li 21 130 0 17 Nov 2017
On Noisy Negative Curvature Descent: Competing with Gradient Descent for Faster Non-convex Optimization Mingrui Liu Tianbao Yang 28 23 0 25 Sep 2017
Natasha: Faster Non-Convex Stochastic Optimization Via Strongly Non-Convex Parameter Zeyuan Allen-Zhu 15 80 0 02 Feb 2017
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods Zeyuan Allen-Zhu ODL 15 575 0 18 Mar 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 84 736 0 19 Mar 2014