Local SGD: Unified Theory and New Efficient Methods

3 November 2020

Papers citing "Local SGD: Unified Theory and New Efficient Methods"

50 / 75 papers shown

Title
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 28 0 0 12 May 2025
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 36 0 0 15 Apr 2025
Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learning Karlo Palenzuela Ali Dadras A. Yurtsever Tommy Löfstedt FedML 40 0 0 27 Mar 2025
A Unified Analysis of Federated Learning with Arbitrary Client Participation Shiqiang Wang Mingyue Ji FedML 29 55 0 31 Dec 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 64 0 0 26 Nov 2024
Tighter Performance Theory of FedExProx Wojciech Anyszka Kaja Gruntkowska A. Tyurin Peter Richtárik FedML 21 0 0 20 Oct 2024
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 27 1 0 10 Oct 2024
Accelerated Stochastic ExtraGradient: Mixing Hessian and Gradient Similarity to Reduce Communication in Distributed and Federated Learning Dmitry Bylinkin Kirill Degtyarev Aleksandr Beznosikov FedML 34 0 0 22 Sep 2024
A survey on secure decentralized optimization and learning Changxin Liu Nicola Bastianello Wei Huo Yang Shi Karl H. Johansson 34 1 0 16 Aug 2024
Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity Qihao Zhou Haishan Ye Luo Luo 24 0 0 25 May 2024
FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models Kai Yi Georg Meinhardt Laurent Condat Peter Richtárik FedML 32 6 0 14 Mar 2024
On the Convergence of Federated Learning Algorithms without Data Similarity Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 33 1 0 29 Feb 2024
SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning Paul Mangold S. Samsonov Safwan Labbi I. Levin Réda Alami Alexey Naumov Eric Moulines 38 1 0 06 Feb 2024
Decomposable Submodular Maximization in Federated Setting Akbar Rafiey FedML 22 1 0 31 Jan 2024
Lotto: Secure Participant Selection against Adversarial Servers in Federated Learning Zhifeng Jiang Peng Ye Shiqi He Wei Wang Ruichuan Chen Bo Li 23 2 0 05 Jan 2024
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms Rustem Islamov M. Safaryan Dan Alistarh FedML 21 12 0 31 Oct 2023
Distributed Personalized Empirical Risk Minimization Yuyang Deng Mohammad Mahdi Kamani Pouria Mahdavinia M. Mahdavi 21 4 0 26 Oct 2023
Over-the-Air Federated Learning and Optimization Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Wei-Neng Chen Khaled B. Letaief FedML 23 11 0 16 Oct 2023
Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization Yan Sun Li Shen Dacheng Tao FedML 20 14 0 09 Jun 2023
A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging Shiqiang Wang Mingyue Ji FedML 30 0 0 06 Jun 2023
Improving Accelerated Federated Learning with Compression and Importance Sampling Michal Grudzieñ Grigory Malinovsky Peter Richtárik FedML 27 8 0 05 Jun 2023
Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning Kai Yi Laurent Condat Peter Richtárik FedML 35 5 0 22 May 2023
DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regime Jongho Park Jinchao Xu FedML 47 1 0 17 May 2023
Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling Nicolò Dal Fabbro A. Mitra George J. Pappas FedML 33 12 0 14 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 43 2 0 09 Apr 2023
Unified analysis of SGD-type methods Eduard A. Gorbunov 22 2 0 29 Mar 2023
TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation Laurent Condat Ivan Agarský Grigory Malinovsky Peter Richtárik FedML 22 4 0 20 Feb 2023
Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity Han Wang A. Mitra Hamed Hassani George J. Pappas James Anderson FedML 24 21 0 04 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 30 5 0 30 Jan 2023
An Optimal Algorithm for Strongly Convex Min-min Optimization Alexander Gasnikov D. Kovalev Grigory Malinovsky 24 1 0 29 Dec 2022
Can 5th Generation Local Training Methods Support Client Sampling? Yes! Michal Grudzieñ Grigory Malinovsky Peter Richtárik 17 28 0 29 Dec 2022
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Federated Hypergradient Descent A. K. Kan FedML 32 3 0 03 Nov 2022
A Convergence Theory for Federated Average: Beyond Smoothness Xiaoxiao Li Zhao-quan Song Runzhou Tao Guangyi Zhang FedML 22 5 0 03 Nov 2022
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 14 6 0 31 Oct 2022
GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity A. Maranjyan M. Safaryan Peter Richtárik 32 13 0 28 Oct 2022
Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$ -Means Clustering Lingxiao Huang Zhize Li Jialin Sun Haoyu Zhao FedML 29 9 0 26 Oct 2022
Provably Doubly Accelerated Federated Learning: The First Theoretically Successful Combination of Local Training and Communication Compression Laurent Condat Ivan Agarský Peter Richtárik FedML 16 16 0 24 Oct 2022
On the Performance of Gradient Tracking with Local Updates Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe 35 18 0 10 Oct 2022
Personalized Federated Learning with Communication Compression El Houcine Bergou Konstantin Burlachenko Aritra Dutta Peter Richtárik FedML 72 9 0 12 Sep 2022
HammingMesh: A Network Topology for Large-Scale Deep Learning Torsten Hoefler Tommaso Bonato Daniele De Sensi Salvatore Di Girolamo Shigang Li Marco Heddes Jon Belk Deepak Goel Miguel Castro Steve Scott 3DH GNN AI4CE 18 20 0 03 Sep 2022
Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization Yan Huang Qihang Lin N. Street Stephen Seung-Yeob Baek FedML 20 9 0 21 Jul 2022
Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning Grigory Malinovsky Kai Yi Peter Richtárik FedML 29 38 0 09 Jul 2022
Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox Abdurakhmon Sadiev D. Kovalev Peter Richtárik 17 20 0 08 Jul 2022
Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology Y. Huang Ying Sun Zehan Zhu Changzhi Yan Jinming Xu FedML 22 15 0 08 Jul 2022
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression Zhize Li Haoyu Zhao Boyue Li Yuejie Chi FedML 22 41 0 20 Jun 2022
Federated Optimization Algorithms with Random Reshuffling and Gradient Compression Abdurakhmon Sadiev Grigory Malinovsky Eduard A. Gorbunov Igor Sokolov Ahmed Khaled Konstantin Burlachenko Peter Richtárik FedML 11 21 0 14 Jun 2022
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation Rustem Islamov Xun Qian Slavomír Hanzely M. Safaryan Peter Richtárik 30 16 0 07 Jun 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 24 75 0 27 May 2022
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks Mingrui Liu Zhenxun Zhuang Yunwei Lei Chunyang Liao 20 16 0 10 May 2022