Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices Huancheng Chen H. Vikalo FedML MQ 16 7 0 29 Nov 2023
Federated Online and Bandit Convex Optimization Kumar Kshitij Patel Lingxiao Wang Aadirupa Saha Nathan Srebro FedML 43 8 0 29 Nov 2023
Asynchronous Wireless Federated Learning with Probabilistic Client Selection Jiarong Yang Yuan Liu Fangjiong Chen Wen Chen Changle Li FedML 25 5 0 28 Nov 2023
Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network Student Member Ieee Hongda Wu F. I. C. V. Ping Wang Aswartha Narayana FedML 44 1 0 16 Nov 2023
DiLoCo: Distributed Low-Communication Training of Language Models Arthur Douillard Qixuang Feng Andrei A. Rusu Rachita Chhaparia Yani Donchev A. Kuncoro MarcÁurelio Ranzato Arthur Szlam Jiajun Shen 56 31 0 14 Nov 2023
EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning Mohammad Mahdi Rahimi Hasnain Irshad Bhatti Younghyun Park Humaira Kousar Jaekyun Moon FedML 38 7 0 13 Nov 2023
Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Data Cheng-Hao Tu Hong-You Chen Zheda Mai Jike Zhong Vardaan Pahuja Tanya Berger-Wolf Song Gao Charles V. Stewart Yu-Chuan Su Wei-Lun Chao CLL 34 3 0 02 Nov 2023
Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated Optimization Mathieu Even Anastasia Koloskova Laurent Massoulié FedML 38 13 0 01 Nov 2023
Distributed Personalized Empirical Risk Minimization Yuyang Deng Mohammad Mahdi Kamani Pouria Mahdavinia M. Mahdavi 21 4 0 26 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 51 1 0 22 Oct 2023
An effective theory of collective deep learning Lluís Arola-Fernández Lucas Lacasa FedML AI4CE 18 2 0 19 Oct 2023
Over-the-Air Federated Learning and Optimization Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Wei-Neng Chen Khaled B. Letaief FedML 23 11 0 16 Oct 2023
Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction Hanhan Zhou Tian-Shing Lan Guru Venkataramani Wenbo Ding 29 28 0 12 Oct 2023
Revisiting Decentralized ProxSkip: Achieving Linear Speedup Luyao Guo Sulaiman A. Alghunaim Kun Yuan Laurent Condat Jinde Cao FedML 36 1 0 12 Oct 2023
Utilizing Free Clients in Federated Learning for Focused Model Enhancement Aditya Narayan Ravi Ilan Shomorony FedML 32 0 0 06 Oct 2023
Stability and Generalization for Minibatch SGD and Local SGD Yunwen Lei Tao Sun Mingrui Liu 29 3 0 02 Oct 2023
Module-wise Training of Neural Networks via the Minimizing Movement Scheme Skander Karkar Bhaskar Sen Emmanuel de Bezenac Patrick Gallinari 25 2 0 29 Sep 2023
FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler Zilinghan Li Pranshu Chaturvedi Shilan He Han-qiu Chen Gagandeep Singh Volodymyr V. Kindratenko Eliu A. Huerta Kibaek Kim Ravi K. Madduri FedML 34 9 0 26 Sep 2023
FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data Hao Sun Li Shen Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao FedML 31 1 0 18 Sep 2023
Convergence Analysis of Decentralized ASGD Mauro Dalle Lucca Tosi Martin Theobald 23 2 0 07 Sep 2023
Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beat Erdong Hu Yu-Shuen Tang Anastasios Kyrillidis C. Jermaine FedML 28 10 0 06 Sep 2023
Composite federated learning with heterogeneous data Jiaojiao Zhang Jiang Hu Mikael Johansson FedML 24 4 0 04 Sep 2023
FedFwd: Federated Learning without Backpropagation Seonghwan Park Dahun Shin Jinseok Chung Namhoon Lee FedML 17 5 0 03 Sep 2023
ABS-SGD: A Delayed Synchronous Stochastic Gradient Descent Algorithm with Adaptive Batch Size for Heterogeneous GPU Clusters Xin Zhou Ling Chen Houming Wu 14 0 0 29 Aug 2023
FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning Gihun Lee Minchan Jeong Sangmook Kim Jaehoon Oh Se-Young Yun FedML 24 8 0 24 Aug 2023
Federated Semi-Supervised and Semi-Asynchronous Learning for Anomaly Detection in IoT Networks Wenbin Zhai Feng Wang L. Liu Youwei Ding Wanyi Lu 27 0 0 23 Aug 2023
ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations Xinpeng Ling Jie Fu Kuncan Wang Haitao Liu Zhili Chen FedML 26 2 0 21 Aug 2023
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 32 195 0 16 Aug 2023
Straggler Mitigation and Latency Optimization in Blockchain-based Hierarchical Federated Learning Zhilin Wang Qin Hu Minghui Xu Zeihui Xiong 37 0 0 02 Aug 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 23 6 0 01 Aug 2023
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 67 4 0 01 Aug 2023
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 19 16 0 30 Jul 2023
Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey Kilian Pfeiffer Martin Rapp R. Khalili J. Henkel FedML 14 63 0 18 Jul 2023
Optimal Compression of Unit Norm Vectors in the High Distortion Regime He Zhu Avishek Ghosh A. Mazumdar 14 3 0 16 Jul 2023
DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates Peyman Gholami H. Seferoglu FedML 10 11 0 14 Jul 2023
Locally Adaptive Federated Learning Sohom Mukherjee Nicolas Loizou Sebastian U. Stich FedML 19 3 0 12 Jul 2023
Benchmarking Algorithms for Federated Domain Generalization Ruqi Bai S. Bagchi David I. Inouye FedML 32 12 0 11 Jul 2023
Momentum Benefits Non-IID Federated Learning Simply and Provably Ziheng Cheng Xinmeng Huang Pengfei Wu Kun Yuan FedML 29 16 0 28 Jun 2023
MimiC: Combating Client Dropouts in Federated Learning by Mimicking Central Updates Yuchang Sun Yuyi Mao Jinchao Zhang FedML 28 9 0 21 Jun 2023
Adaptive Federated Learning with Auto-Tuned Clients J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis FedML 40 6 0 19 Jun 2023
Differentially Private Over-the-Air Federated Learning Over MIMO Fading Channels Hang Liu Jiahe Yan Y. Zhang 21 3 0 19 Jun 2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 25 2 0 18 Jun 2023
Distributed Semi-Supervised Sparse Statistical Inference Jiyuan Tu Weidong Liu Xiaojun Mao Mingyue Xu 19 1 0 17 Jun 2023
$$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning$ $\textbf{A}^2\textbf{CiD}^2$ : Accelerating Asynchronous Communication in Decentralized Deep Learning Adel Nabli Eugene Belilovsky Edouard Oyallon 19 6 0 14 Jun 2023
Differentially Private Wireless Federated Learning Using Orthogonal Sequences Xizixiang Wei Tianhao Wang Ruiquan Huang Cong Shen Jing Yang H. Vincent Poor 43 1 0 14 Jun 2023
Protecting User Privacy in Remote Conversational Systems: A Privacy-Preserving framework based on text sanitization Zhigang Kan Linbo Qiao Hao Yu Liwen Peng Yifu Gao Dongsheng Li 26 20 0 14 Jun 2023
On the Computation-Communication Trade-Off with A Flexible Gradient Tracking Approach Yan Huang Jinming Xu 25 2 0 12 Jun 2023
FedDec: Peer-to-peer Aided Federated Learning Marina Costantini Giovanni Neglia T. Spyropoulos FedML 8 1 0 11 Jun 2023
Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often! Milad Sefidgaran Romain Chor A. Zaidi Yijun Wan FedML 24 6 0 09 Jun 2023
Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates Siqi Zhang S. Choudhury Sebastian U. Stich Nicolas Loizou FedML 19 3 0 08 Jun 2023