Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping

21 May 2020

Alexander Gasnikov

Papers citing "Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping"

26 / 26 papers shown

Title
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees Aleksandar Armacki Shuhua Yu Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 57 2 0 17 Oct 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
Differential Private Stochastic Optimization with Heavy-tailed Data: Towards Optimal Rates Puning Zhao Jiafei Wu Zhe Liu Chong Wang Rongfei Fan Qingming Li 45 1 0 19 Aug 2024
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping Umut Simsekli Mert Gurbuzbalaban S. Yıldırım Lingjiong Zhu 38 2 0 04 Mar 2024
AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size P. Ostroukhov Aigerim Zhumabayeva Chulu Xiang Alexander Gasnikov Martin Takáč Dmitry Kamzolov ODL 43 2 0 07 Feb 2024
General Tail Bounds for Non-Smooth Stochastic Mirror Descent Khaled Eldowa Andrea Paudice 16 4 0 12 Dec 2023
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 39 2 0 20 Nov 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 32 10 0 30 May 2023
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation Zijian Liu Zhengyuan Zhou 24 10 0 22 Mar 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 32 12 0 14 Feb 2023
U-Clip: On-Average Unbiased Stochastic Gradient Clipping Bryn Elesedy Marcus Hutter 11 1 0 06 Feb 2023
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 27 12 0 03 Oct 2022
Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems: A Survey Aleksandr Beznosikov Boris Polyak Eduard A. Gorbunov D. Kovalev Alexander Gasnikov 37 31 0 29 Aug 2022
High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise D. A. Parletta Andrea Paudice Massimiliano Pontil Saverio Salzo 34 9 0 17 Aug 2022
Learning from time-dependent streaming data with online stochastic algorithms Antoine Godichon-Baggioni Nicklas Werge Olivier Wintenberger 22 3 0 25 May 2022
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms Binghui Xie Chen Jin Kaiwen Zhou James Cheng Wei Meng 35 1 0 28 Apr 2022
Nonlinear gradient mappings and stochastic optimization: A general framework with applications to heavy-tail noise D. Jakovetić Dragana Bajović Anit Kumar Sahu S. Kar Nemanja Milošević Dusan Stamenkovic 17 12 0 06 Apr 2022
Flexible risk design using bi-directional dispersion Matthew J. Holland 32 5 0 28 Mar 2022
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance Hongjian Wang Mert Gurbuzbalaban Lingjiong Zhu Umut cSimcsekli Murat A. Erdogdu 15 41 0 20 Feb 2021
Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness Vien V. Mai M. Johansson 23 38 0 12 Feb 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 32 2 0 04 Jan 2021
First-Order Methods for Convex Optimization Pavel Dvurechensky Mathias Staudigl Shimrit Shtern ODL 18 25 0 04 Jan 2021
Learning from History for Byzantine Robust Optimization Sai Praneeth Karimireddy Lie He Martin Jaggi FedML AAML 22 173 0 18 Dec 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 31 79 0 17 Sep 2020
Sharp Concentration Results for Heavy-Tailed Distributions Milad Bakhshizadeh A. Maleki Víctor Pena 9 21 0 30 Mar 2020