Lower Bounds for Non-Convex Stochastic Optimization

5 December 2019

Papers citing "Lower Bounds for Non-Convex Stochastic Optimization"

50 / 71 papers shown

Title
Observability conditions for neural state-space models with eigenvalues and their roots of unity Andrew Gracyk 120 0 0 22 Apr 2025
Nested Stochastic Gradient Descent for (Generalized) Sinkhorn Distance-Regularized Distributionally Robust Optimization Y. Yang Yi Zhou Zhaosong Lu 49 0 0 29 Mar 2025
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 54 0 0 16 Mar 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 100 0 0 21 Feb 2025
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 130 0 0 10 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 61 0 0 08 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 43 2 0 28 Sep 2024
Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models Matteo Lapucci Davide Pucci 35 1 0 06 Aug 2024
Private Zeroth-Order Nonsmooth Nonconvex Optimization Qinzi Zhang Hoang Tran Ashok Cutkosky 40 4 0 27 Jun 2024
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum H. Cai Sulaiman A. Alghunaim Ali H.Sayed 43 1 0 18 Jun 2024
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 35 4 0 16 May 2024
Learning Optimal Deterministic Policies with Stochastic Policy Gradients Alessandro Montenegro Marco Mussi Alberto Maria Metelli Matteo Papini 42 2 0 03 May 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 62 1 0 03 Apr 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 34 3 0 01 Apr 2024
Stochastic Halpern iteration in normed spaces and applications to reinforcement learning Mario Bravo Juan Pablo Contreras 40 3 0 19 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 29 6 0 05 Mar 2024
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad Sayantan Choudhury N. Tupitsa Nicolas Loizou Samuel Horváth Martin Takáč Eduard A. Gorbunov 30 1 0 05 Mar 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 46 10 0 06 Feb 2024
How Free is Parameter-Free Stochastic Optimization? Amit Attia Tomer Koren ODL 44 4 0 05 Feb 2024
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis Jie Hao Xiaochuan Gong Mingrui Liu 25 7 0 17 Jan 2024
Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications R. Karandikar M. Vidyasagar 25 8 0 05 Dec 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 26 0 0 19 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 40 5 0 15 Oct 2023
Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems Gabriel Mancino-Ball Yangyang Xu 18 8 0 14 Jul 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 25 8 0 26 Jun 2023
Symmetry & Critical Points for Symmetric Tensor Decomposition Problems Yossi Arjevani Gal Vinograd 26 5 0 13 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 32 6 0 25 May 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods Junchi Yang Xiang Li Ilyas Fatkhullin Niao He 36 15 0 21 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 28 7 0 12 May 2023
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold S. Schechtman D. Tiapkin Michael Muehlebach Eric Moulines 27 6 0 16 Mar 2023
Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization Feihu Huang Chunyu Xuan Xinrui Wang Siqi Zhang Songcan Chen 28 7 0 07 Mar 2023
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization Tesi Xiao Xuxing Chen Krishnakumar Balasubramanian Saeed Ghadimi 28 10 0 20 Feb 2023
Solving stochastic weak Minty variational inequalities without increasing batch size Thomas Pethick Olivier Fercoq P. Latafat Panagiotis Patrinos V. Cevher 13 23 0 17 Feb 2023
SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance Amit Attia Tomer Koren ODL 17 24 0 17 Feb 2023
Deterministic Nonsmooth Nonconvex Optimization Michael I. Jordan Guy Kornowski Tianyi Lin Ohad Shamir Manolis Zampetakis 49 24 0 16 Feb 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 30 12 0 14 Feb 2023
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization Zichong Li Pinzhuo Chen Sijia Liu Songtao Lu Yangyang Xu 27 17 0 19 Dec 2022
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 24 3 0 12 Dec 2022
Momentum Aggregation for Private Non-convex ERM Hoang Tran Ashok Cutkosky 18 14 0 12 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 27 12 0 03 Oct 2022
Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization Tianyi Lin Zeyu Zheng Michael I. Jordan 49 51 0 12 Sep 2022
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization Wei Jiang Gang Li Yibo Wang Lijun Zhang Tianbao Yang 27 16 0 18 Jul 2022
Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization Quanqi Hu Yongjian Zhong Tianbao Yang 25 16 0 01 Jun 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 21 47 0 09 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 23 8 0 18 Feb 2022
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization Wei Jiang Bokun Wang Yibo Wang Lijun Zhang Tianbao Yang 74 17 0 15 Feb 2022
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 25 1 0 25 Oct 2021