Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning

Neural Information Processing Systems (NeurIPS), 2022

16 June 2022

Papers citing "Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning"

50 / 63 papers shown

Title
Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching Mudit Gaur Prashant Trivedi Shuchin Aeron Amrit Singh Bedi George K. Atia Vaneet Aggarwal 20 0 0 01 Dec 2025
Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning Yujia Wang Fenglong Ma Jinghui Chen FedML 266 0 0 25 Nov 2025
PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture Yi Liu Yang Liu Leqian Zheng Jue Hong Junjie Shi Qingyou Yang Ye Wu Cong Wang 92 0 0 14 Oct 2025
Discrete State Diffusion Models: A Sample Complexity Perspective Aadithya Srikanth Mudit Gaur Vaneet Aggarwal DiffM 85 0 0 12 Oct 2025
FedMuon: Federated Learning with Bias-corrected LMO-based Optimization Yuki Takezawa Anastasia Koloskova Xiaowen Jiang Sebastian U. Stich 142 0 0 30 Sep 2025
Asynchronous Policy Gradient Aggregation for Efficient Distributed Reinforcement Learning Alexander Tyurin Andrei Spiridonov Varvara Rudenko OffRL 108 0 0 29 Sep 2025
Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation Kaoru Otsuka Yuki Takezawa Makoto Yamada FedML 131 1 0 03 Sep 2025
Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Training Rongwei Lu Jingyan Jiang Chunyang Li Haotian Dong Xingguang Wei 157 1 0 23 Jul 2025
Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access Mudit Gaur Prashant Trivedi Sasidhar Kunapuli Amrit Singh Bedi Vaneet Aggarwal 466 0 0 23 May 2025
$γ$ -FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated LearningIEEE Conference on Computer Communications (IEEE INFOCOM), 2025 Rongwei Lu Yutong Jiang Jinrui Zhang Chunyang Li Yifei Zhu Bin Chen Zhi Wang FedML 316 2 0 18 May 2025
Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework Yijie Zhou Shi Pu 141 1 0 15 May 2025
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao FedML 298 0 0 28 Apr 2025
A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip Peyman Gholami H. Seferoglu 139 0 0 14 Apr 2025
Orthogonal Calibration for Asynchronous Federated Learning Jiayun Zhang Shuheng Li Haiyu Huang Xiaofan Yu Rajesh K. Gupta Jingbo Shang FedML 222 1 0 21 Feb 2025
Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity Artavazd Maranjyan Alexander Tyurin Peter Richtárik 337 6 0 27 Jan 2025
Review of Mathematical Optimization in Federated Learning Shusen Yang Fangyuan Zhao Zihao Zhou Liang Shi Xuebin Ren Zongben Xu FedML AI4CE 311 6 0 02 Dec 2024
Boosting Asynchronous Decentralized Learning with Model FragmentationThe Web Conference (WWW), 2024 Sayan Biswas Anne-Marie Kermarrec Alexis Marouani Rafael Pires Rishi Sharma M. Vos 220 5 0 16 Oct 2024
MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute TimesConference on Uncertainty in Artificial Intelligence (UAI), 2024 Artavazd Maranjyan Omar Shaikh Omar Peter Richtárik 206 4 0 05 Oct 2024
Efficient Federated Learning against Heterogeneous and Non-stationary Client UnavailabilityNeural Information Processing Systems (NeurIPS), 2024 Ming Xiang Stratis Ioannidis Edmund Yeh Carlee Joe-Wong Lili Su FedML 271 8 0 26 Sep 2024
Sequential Federated Learning in Hierarchical Architecture on Non-IID DatasetsIEEE Transactions on Mobile Computing (IEEE TMC), 2024 Xingrun Yan Shiyuan Zuo Rongfei Fan Han Hu Li Shen Puning Zhao Yong Luo FedML 204 0 0 19 Aug 2024
Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching Amit Attia Ofir Gaash Tomer Koren 262 0 0 14 Aug 2024
Finite-Time Analysis of Asynchronous Multi-Agent TD LearningAmerican Control Conference (ACC), 2024 Nicolò Dal Fabbro Arman Adibi Aritra Mitra George J. Pappas 197 2 0 29 Jul 2024
Ordered Momentum for Asynchronous SGDNeural Information Processing Systems (NeurIPS), 2024 Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 388 2 0 27 Jul 2024
FADAS: Towards Federated Adaptive Asynchronous Optimization Yujia Wang Shiqiang Wang Songtao Lu Jinghui Chen FedML 185 11 0 25 Jul 2024
A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning Jiajun Song Jiajun Luo Rongwei Lu Shuzhao Xie Bin Chen Zhi Wang FedML 122 2 0 06 Jul 2024
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 288 3 0 20 Jun 2024
FedAST: Federated Asynchronous Simultaneous Training Baris Askin Pranay Sharma Carlee Joe-Wong Gauri Joshi 262 6 0 01 Jun 2024
Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data Xiaolu Wang Yuchang Sun Hoi-To Wai Jun Zhang 240 2 0 27 May 2024
Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations Alexander Tyurin Kaja Gruntkowska Peter Richtárik 236 6 0 24 May 2024
Asynchronous Federated Stochastic Optimization for Heterogeneous Objectives Under Arbitrary Delays Charikleia Iakovidou Kibaek Kim FedML 324 3 0 16 May 2024
Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client ParticipationInternational Conference on Machine Learning (ICML), 2024 Haibo Yang Pei-Yuan Qiu Prashant Khanduri Minghong Fang Jia Liu FedML 218 5 0 04 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 307 20 0 09 Apr 2024
Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis Guangchen Lan Dong-Jun Han Abolfazl Hashemi Vaneet Aggarwal Christopher G. Brinton 752 22 0 09 Apr 2024
DASA: Delay-Adaptive Multi-Agent Stochastic Approximation Nicolò Dal Fabbro Arman Adibi H. Vincent Poor Sanjeev R. Kulkarni A. Mitra George J. Pappas 196 3 0 25 Mar 2024
Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling Arman Adibi Nicolò Dal Fabbro Luca Schenato Sanjeev R. Kulkarni H. Vincent Poor George J. Pappas Hamed Hassani A. Mitra 307 9 0 19 Feb 2024
Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients Xiaolu Wang Zijian Li Shi Jin Jun Zhang FedML 324 6 0 17 Feb 2024
Queuing dynamics of asynchronous Federated Learning Louis Leconte Matthieu Jonckheere S. Samsonov Eric Moulines FedML 203 8 0 12 Feb 2024
CO2: Efficient Distributed Training with Full Communication-Computation Overlap Weigao Sun Zhen Qin Weixuan Sun Shidi Li Dong Li Xuyang Shen Yu Qiao Yiran Zhong OffRL 222 14 0 29 Jan 2024
EchoPFL: Asynchronous Personalized Federated Learning on Mobile Devices with On-Demand Staleness ControlProceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (IMWUT), 2024 Xiaocheng Li Si-ren Liu Zimu Zhou Bin Guo Yuan Xu Zhiwen Yu 255 2 0 29 Jan 2024
Efficient Asynchronous Federated Learning with Sparsification and Quantization Juncheng Jia Ji Liu Chendi Zhou Hao Tian M. Dong Dejing Dou FedML 289 17 0 23 Dec 2023
FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model UpdateAAAI Conference on Artificial Intelligence (AAAI), 2023 Ji Liu Juncheng Jia Tianshi Che Chao Huo Jiaxiang Ren Yang Zhou H. Dai Dejing Dou 215 66 0 10 Dec 2023
Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated OptimizationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Mathieu Even Anastasia Koloskova Laurent Massoulié FedML 308 16 0 01 Nov 2023
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD AlgorithmsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Rustem Islamov M. Safaryan Dan Alistarh FedML 204 22 0 31 Oct 2023
Convergence Analysis of Decentralized ASGD Mauro Dalle Lucca Tosi Martin Theobald 223 3 0 07 Sep 2023
Non-ergodic linear convergence property of the delayed gradient descent under the strongly convexity and the Polyak-Łojasiewicz conditionAnalysis and Applications (AA), 2023 Hyunggwon Choi Woocheol Choi Jinmyoung Seok 141 0 0 23 Aug 2023
Towards Understanding the Generalizability of Delayed Stochastic Gradient DescentIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Xiaoge Deng Li Shen Shengwei Li Tao Sun Dongsheng Li Dacheng Tao 296 3 0 18 Aug 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 165 11 0 01 Aug 2023
Min-Max Optimization under DelaysAmerican Control Conference (ACC), 2023 Arman Adibi A. Mitra Hamed Hassani 253 2 0 13 Jul 2023
Locally Adaptive Federated Learning Sohom Mukherjee Nicolas Loizou Sebastian U. Stich FedML 139 7 0 12 Jul 2023
Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity Konstantin Mishchenko Rustem Islamov Eduard A. Gorbunov Samuel Horváth FedML 224 12 0 29 May 2023