Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization

30 October 2019

Farzin Haddadpour

Mohammad Mahdi Kamani

Papers citing "Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization"

35 / 35 papers shown

Title
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 63 0 0 25 Apr 2025
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 30 1 0 22 Jul 2024
Distributed Personalized Empirical Risk Minimization Yuyang Deng Mohammad Mahdi Kamani Pouria Mahdavinia M. Mahdavi 23 4 0 26 Oct 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 28 0 0 01 Aug 2023
Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching S. Tyagi Prateek Sharma 16 22 0 20 May 2023
Federated Gradient Matching Pursuit Halyun Jeong Deanna Needell Jing Qin FedML 35 1 0 20 Feb 2023
Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity Han Wang A. Mitra Hamed Hassani George J. Pappas James Anderson FedML 26 21 0 04 Feb 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization Heting Liu Fang He Guohong Cao FedML MQ 18 24 0 16 Dec 2022
Communication-Efficient Local SGD with Age-Based Worker Selection Feng Zhu Jingjing Zhang Xin Eric Wang 22 1 0 31 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Eric Wang 26 3 0 06 Oct 2022
Fast Heterogeneous Federated Learning with Hybrid Client Selection Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 27 12 0 10 Aug 2022
Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates Fangcheng Fu Xupeng Miao Jiawei Jiang Huanran Xue Bin Cui FedML 22 21 0 29 Jul 2022
Confederated Learning: Federated Learning with Decentralized Edge Servers Bin Wang Jun Fang Hongbin Li Xiaojun Yuan Qing Ling FedML 21 23 0 30 May 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 26 75 0 27 May 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 27 3 0 22 Mar 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 20 2 0 21 Mar 2022
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 32 71 0 27 Oct 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 23 30 0 03 Jul 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 30 288 0 11 Jun 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 27 58 0 25 Feb 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 55 157 0 14 Feb 2021
Design and Analysis of Uplink and Downlink Communications for Federated Learning Sihui Zheng Cong Shen Xiang Chen 28 139 0 07 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 29 57 0 17 Nov 2020
Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 28 51 0 24 Oct 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 14 11 0 23 Jul 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 14 1,295 0 15 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 30 271 0 02 Jul 2020
Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks Zhishuai Guo Mingrui Liu Zhuoning Yuan Li Shen Wei Liu Tianbao Yang 24 42 0 05 May 2020
Adaptive Personalized Federated Learning Yuyang Deng Mohammad Mahdi Kamani M. Mahdavi FedML 212 542 0 30 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 22 12 0 06 Mar 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
On the Convergence of Local Descent Methods in Federated Learning Farzin Haddadpour M. Mahdavi FedML 19 265 0 31 Oct 2019
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 136 1,198 0 16 Aug 2016