Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
Trends and Advancements in Deep Neural Network Communication Felix Sattler Thomas Wiegand Wojciech Samek GNN 19 9 0 06 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 17 12 0 06 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 10 1,387 0 29 Feb 2020
Moniqua: Modulo Quantized Communication in Decentralized SGD Yucheng Lu Christopher De Sa MQ 19 50 0 26 Feb 2020
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Zhize Li D. Kovalev Xun Qian Peter Richtárik FedML AI4CE 18 133 0 26 Feb 2020
LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning Tianyi Chen Yuejiao Sun W. Yin FedML 6 14 0 26 Feb 2020
Device Heterogeneity in Federated Learning: A Superquantile Approach Yassine Laguel Krishna Pillutla J. Malick Zaïd Harchaoui FedML 32 22 0 25 Feb 2020
Network-Density-Controlled Decentralized Parallel Stochastic Gradient Descent in Wireless Systems Koya Sato Yasuyuki Satoh D. Sugimura 11 1 0 25 Feb 2020
Three Approaches for Personalization with Applications to Federated Learning Yishay Mansour M. Mohri Jae Hun Ro A. Suresh FedML 34 565 0 25 Feb 2020
Quantized Decentralized Stochastic Learning over Directed Graphs Hossein Taheri Aryan Mokhtari Hamed Hassani Ramtin Pedarsani 6 5 0 23 Feb 2020
Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD Jianyu Wang Hao Liang Gauri Joshi 6 33 0 21 Feb 2020
Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor M. Safaryan Egor Shulgin Peter Richtárik 24 60 0 20 Feb 2020
Dynamic Federated Learning Elsa Rizk Stefan Vlaski A. H. Sayed FedML 6 25 0 20 Feb 2020
Communication-Efficient Distributed SVD via Local Power Iterations Xiang Li Shusen Wang Kun Chen Zhihua Zhang 27 21 0 19 Feb 2020
Personalized Federated Learning: A Meta-Learning Approach Alireza Fallah Aryan Mokhtari Asuman Ozdaglar FedML 31 560 0 19 Feb 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
Distributed Optimization over Block-Cyclic Data Yucheng Ding Chaoyue Niu Yikai Yan Zhenzhe Zheng Fan Wu Guihai Chen Shaojie Tang Rongfei Jia FedML 63 15 0 18 Feb 2020
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability Yikai Yan Chaoyue Niu Yucheng Ding Zhenzhe Zheng Fan Wu Guihai Chen Shaojie Tang Zhihua Wu FedML 36 37 0 18 Feb 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 55 30 0 13 Feb 2020
Federated Learning of a Mixture of Global and Local Models Filip Hanzely Peter Richtárik FedML 4 378 0 10 Feb 2020
Faster On-Device Training Using New Federated Momentum Algorithm Zhouyuan Huo Qian Yang Bin Gu Heng-Chiao Huang FedML 9 47 0 06 Feb 2020
Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge M. Amiri Deniz Gunduz Sanjeev R. Kulkarni H. Vincent Poor 75 170 0 28 Jan 2020
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent Giorgi Nadiradze Ilia Markov Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 6 14 0 16 Jan 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 30 55 0 07 Jan 2020
FedDANE: A Federated Newton-Type Method Tian Li Anit Kumar Sahu Manzil Zaheer Maziar Sanjabi Ameet Talwalkar Virginia Smith FedML 18 155 0 07 Jan 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 24 152 0 30 Dec 2019
Parallel Restarted SPIDER -- Communication Efficient Distributed Nonconvex Optimization with Optimal Computation Complexity Pranay Sharma Swatantra Kafle Prashant Khanduri Saikiran Bulusu K. Rajawat P. Varshney FedML 15 17 0 12 Dec 2019
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 69 6,063 0 10 Dec 2019
Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates Cong Xie Oluwasanmi Koyejo Indranil Gupta Haibin Lin 16 39 0 20 Nov 2019
A Crowdsourcing Framework for On-Device Federated Learning Shashi Raj Pandey N. H. Tran M. Bennis Y. Tun Aunas Manzoor C. Hong FedML 12 250 0 04 Nov 2019
Device Scheduling with Fast Convergence for Wireless Federated Learning Misha Sra C. Schmandt Z. Niu FedML 14 189 0 03 Nov 2019
On the Convergence of Local Descent Methods in Federated Learning Farzin Haddadpour M. Mahdavi FedML 19 265 0 31 Oct 2019
SPARQ-SGD: Event-Triggered and Compressed Communication in Decentralized Stochastic Optimization Navjot Singh Deepesh Data Jemin George Suhas Diggavi 13 23 0 31 Oct 2019
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization Farzin Haddadpour Mohammad Mahdi Kamani M. Mahdavi V. Cadambe FedML 12 199 0 30 Oct 2019
Asynchronous Decentralized SGD with Quantized and Local Updates Giorgi Nadiradze Amirmojtaba Sabour Peter Davies Shigang Li Dan Alistarh 9 49 0 27 Oct 2019
Sparsification as a Remedy for Staleness in Distributed Asynchronous SGD Rosa Candela Giulio Franzese Maurizio Filippone Pietro Michiardi 8 1 0 21 Oct 2019
Communication-Efficient Local Decentralized SGD Methods Xiang Li Wenhao Yang Shusen Wang Zhihua Zhang 16 53 0 21 Oct 2019
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 17 342 0 14 Oct 2019
Model Fusion via Optimal Transport Sidak Pal Singh Martin Jaggi MoMe FedML 15 225 0 12 Oct 2019
Central Server Free Federated Learning over Single-sided Trust Social Networks Chaoyang He Conghui Tan Hanlin Tang Shuang Qiu Ji Liu FedML 10 73 0 11 Oct 2019
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum Jianyu Wang Vinayak Tantia Nicolas Ballas Michael G. Rabbat 4 200 0 01 Oct 2019
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ali Jadbabaie Ramtin Pedarsani FedML 162 760 0 28 Sep 2019
Matrix Sketching for Secure Collaborative Machine Learning Mengjiao Zhang Shusen Wang FedML 8 14 0 24 Sep 2019
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication Sebastian U. Stich Sai Praneeth Karimireddy FedML 12 20 0 11 Sep 2019
Addressing Algorithmic Bottlenecks in Elastic Machine Learning with Chicle Michael Kaufmann K. Kourtis Celestine Mendler-Dünner Adrian Schüpbach Thomas Parnell 6 0 0 11 Sep 2019
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 13 424 0 10 Sep 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 16 22 0 10 Sep 2019
First Analysis of Local GD on Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik FedML 11 172 0 10 Sep 2019
Federated Learning: Challenges, Methods, and Future Directions Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 11 4,413 0 21 Aug 2019
Federated Learning over Wireless Fading Channels M. Amiri Deniz Gunduz 22 505 0 23 Jul 2019