Momentum Provably Improves Error Feedback!

24 May 2023

Papers citing "Momentum Provably Improves Error Feedback!"

22 / 22 papers shown

Title
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 43 0 0 11 Mar 2025
Smoothed Normalization for Efficient Distributed Private Optimization Egor Shulgin Sarit Khirirat Peter Richtárik FedML 82 0 0 20 Feb 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 28 0 0 11 Nov 2024
Error Feedback under $(L_0,L_1)$ -Smoothness: Normalization and Momentum Sarit Khirirat Abdurakhmon Sadiev Artem Riabinin Eduard A. Gorbunov Peter Richtárik 20 0 0 22 Oct 2024
Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering Changxin Liu Yanghao Li Yuhao Yi Karl H. Johansson FedML 26 0 0 13 Sep 2024
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training Xingyu Xie Zhijie Lin Kim-Chuan Toh Pan Zhou 21 2 0 05 Jul 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 21 6 0 05 Mar 2024
Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergence Ilyas Fatkhullin Niao He 27 3 0 27 Feb 2024
EControl: Fast Distributed Optimization with Compression and Error Control Yuan Gao Rustem Islamov Sebastian U. Stich 25 6 0 06 Nov 2023
Revisiting Decentralized ProxSkip: Achieving Linear Speedup Luyao Guo Sulaiman A. Alghunaim Kun Yuan Laurent Condat Jinde Cao FedML 19 1 0 12 Oct 2023
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 30 194 0 16 Aug 2023
Momentum Benefits Non-IID Federated Learning Simply and Provably Ziheng Cheng Xinmeng Huang Pengfei Wu Kun Yuan FedML 21 16 0 28 Jun 2023
Towards a Better Theoretical Understanding of Independent Subnetwork Training Egor Shulgin Peter Richtárik AI4CE 16 6 0 28 Jun 2023
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression Kaja Gruntkowska A. Tyurin Peter Richtárik 36 21 0 30 Sep 2022
DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronization A. Tyurin Peter Richtárik 32 17 0 02 Feb 2022
Permutation Compressors for Provably Faster Distributed Nonconvex Optimization Rafal Szlendak A. Tyurin Peter Richtárik 113 35 0 07 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 42 44 0 07 Oct 2021
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs Satyen Kale Ayush Sekhari Karthik Sridharan 173 29 0 11 Jul 2021
Linearly Converging Error Compensated SGD Eduard A. Gorbunov D. Kovalev Dmitry Makarenko Peter Richtárik 163 77 0 23 Oct 2020
A High Probability Analysis of Adaptive SGD with Momentum Xiaoyun Li Francesco Orabona 79 64 0 28 Jul 2020
The Future of Digital Health with Federated Learning Nicola Rieke Jonny Hancox Wenqi Li Fausto Milletari H. Roth ... Ronald M. Summers Andrew Trask Daguang Xu Maximilian Baust M. Jorge Cardoso OOD 174 1,698 0 18 Mar 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016