Improved Analysis of Clipping Algorithms for Non-convex Optimization

Improved Analysis of Clipping Algorithms for Non-convex Optimization

5 October 2020

Cong Fang

Papers citing "Improved Analysis of Clipping Algorithms for Non-convex Optimization"

18 / 18 papers shown

Title
An Improved Privacy and Utility Analysis of Differentially Private SGD with Bounded Domain and Smooth Losses Hao Liang W. Zhang Xinlei He Kaishun He Hong Xing 47 0 0 25 Feb 2025
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 61 0 0 08 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 38 2 0 28 Sep 2024
Recent Advances in Non-convex Smoothness Conditions and Applicability to Deep Linear Neural Networks Vivak Patel Christian Varner 26 0 0 20 Sep 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 27 3 0 01 Apr 2024
Directional Smoothness and Gradient Methods: Convergence and Adaptivity Aaron Mishkin Ahmed Khaled Yuanhao Wang Aaron Defazio Robert Mansel Gower 36 6 0 06 Mar 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 40 10 0 06 Feb 2024
Regularized Q-Learning with Linear Function Approximation Jiachen Xi Alfredo Garcia P. Momcilovic 27 2 0 26 Jan 2024
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis Jie Hao Xiaochuan Gong Mingrui Liu 25 7 0 17 Jan 2024
Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping Zijie Pan Jiachen Lu Xiatian Zhu Li Zhang DiffM 28 11 0 19 Oct 2023
Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions Bo Wang Huishuai Zhang Zhirui Ma Wei Chen 32 48 0 29 May 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression Boyue Li Yuejie Chi 21 12 0 17 May 2023
U-Clip: On-Average Unbiased Stochastic Gradient Clipping Bryn Elesedy Marcus Hutter 11 1 0 06 Feb 2023
Variants of SGD for Lipschitz Continuous Loss Functions in Low-Precision Environments Michael R. Metel 25 1 0 09 Nov 2022
RBGNet: Ray-based Grouping for 3D Object Detection Haiyang Wang Shaoshuai Shi Ze Yang Rongyao Fang Qishu Qian Hongsheng Li Bernt Schiele Liwei Wang 3DPC 32 58 0 05 Apr 2022
Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis Jikai Jin Samir Bhatt Haiyang Wang Liwei Wang 25 46 0 24 Oct 2021
Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness Vien V. Mai M. Johansson 16 38 0 12 Feb 2021
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 76 129 0 16 Oct 2018