SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

4 July 2018

Cong Fang

C. J. Li

Zhouchen Lin

Tong Zhang

ArXiv PDF HTML

Papers citing "SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator"

50 / 78 papers shown

Title
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 96 0 0 21 Feb 2025
A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization Wei Liu Yangyang Xu 51 3 0 31 Jan 2025
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 100 0 0 10 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 56 0 0 08 Oct 2024
Obtaining Lower Query Complexities through Lightweight Zeroth-Order Proximal Gradient Algorithms Bin Gu Xiyuan Wei Hualin Zhang Yi Chang Heng-Chiao Huang FedML 21 0 0 03 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 31 2 0 28 Sep 2024
Private Zeroth-Order Nonsmooth Nonconvex Optimization Qinzi Zhang Hoang Tran Ashok Cutkosky 38 4 0 27 Jun 2024
Comparisons Are All You Need for Optimizing Smooth Functions Chenyi Zhang Tongyang Li AAML 24 1 0 19 May 2024
Robust Decentralized Learning with Local Updates and Gradient Tracking Sajjad Ghiasvand Amirhossein Reisizadeh Mahnoosh Alizadeh Ramtin Pedarsani 28 3 0 02 May 2024
Stochastic Halpern iteration in normed spaces and applications to reinforcement learning Mario Bravo Juan Pablo Contreras 38 3 0 19 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 21 6 0 05 Mar 2024
Probabilistic Guarantees of Stochastic Recursive Gradient in Non-Convex Finite Sum Problems Yanjie Zhong Jiaqi Li Soumendra Lahiri 22 1 0 29 Jan 2024
A Coefficient Makes SVRG Effective Yida Yin Zhiqiu Xu Zhiyuan Li Trevor Darrell Zhuang Liu 20 1 0 09 Nov 2023
Adaptive Mirror Descent Bilevel Optimization Feihu Huang 28 1 0 08 Nov 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 35 5 0 15 Oct 2023
Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach Leonardo F. Toso Han Wang James Anderson 27 2 0 19 Sep 2023
Achieving Linear Speedup in Decentralized Stochastic Compositional Minimax Optimization Hongchang Gao 30 1 0 25 Jul 2023
Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems Gabriel Mancino-Ball Yangyang Xu 18 7 0 14 Jul 2023
Efficient preconditioned stochastic gradient descent for estimation in latent variable models C. Baey Maud Delattre E. Kuhn Jean-Benoist Léger Sarah Lemler 6 4 0 22 Jun 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 24 0 0 02 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 22 6 0 25 May 2023
On Momentum-Based Gradient Methods for Bilevel Optimization with Nonconvex Lower-Level Feihu Huang 19 18 0 07 Mar 2023
Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks Arun Ganesh Daogao Liu Sewoong Oh Abhradeep Thakurta ODL 17 12 0 20 Feb 2023
Solving stochastic weak Minty variational inequalities without increasing batch size Thomas Pethick Olivier Fercoq P. Latafat Panagiotis Patrinos V. Cevher 13 23 0 17 Feb 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 30 12 0 14 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 13 1 0 28 Jan 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 25 6 0 09 Jan 2023
Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization G. Fort Eric Moulines 44 6 0 02 Jan 2023
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization Zichong Li Pinzhuo Chen Sijia Liu Songtao Lu Yangyang Xu 13 17 0 19 Dec 2022
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 19 3 0 12 Dec 2022
Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems Hongchang Gao 16 16 0 06 Dec 2022
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods Yanli Liu K. Zhang Tamer Basar W. Yin 30 102 0 15 Nov 2022
SARAH-based Variance-reduced Algorithm for Stochastic Finite-sum Cocoercive Variational Inequalities Aleksandr Beznosikov Alexander Gasnikov 25 2 0 12 Oct 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 22 7 0 04 Oct 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 16 17 0 29 Sep 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 8 14 0 17 Aug 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 21 1 0 17 Aug 2022
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization Wei Jiang Gang Li Yibo Wang Lijun Zhang Tianbao Yang 25 16 0 18 Jul 2022
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 19 2 0 17 Jul 2022
Quantum Neural Network Compression Zhirui Hu Peiyan Dong Zhepeng Wang Youzuo Lin Yanzhi Wang Weiwen Jiang GNN 25 28 0 04 Jul 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 18 1 0 28 Apr 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 33 178 0 28 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 16 8 0 18 Feb 2022
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization Wei Jiang Bokun Wang Yibo Wang Lijun Zhang Tianbao Yang 74 17 0 15 Feb 2022
Decentralized Stochastic Variance Reduced Extragradient Method Luo Luo Haishan Ye 16 7 0 01 Feb 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin R. L. Jin Tianbao Yang 39 46 0 07 Dec 2021
Federated Expectation Maximization with heterogeneity mitigation and variance reduction Aymeric Dieuleveut G. Fort Eric Moulines Geneviève Robin FedML 23 5 0 03 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 25 1 0 25 Oct 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang K. Zhang Tamer Bacsar 23 57 0 12 Oct 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 69 62 0 23 Jul 2021