Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping Junya Chen Sijia Wang Lawrence Carin Chenyang Tao 6 3 0 04 Nov 2021
A Personalized Federated Learning Algorithm: an Application in Anomaly Detection Ali Anaissi Basem Suleiman FedML 19 1 0 04 Nov 2021
Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning Xun Qian Rustem Islamov M. Safaryan Peter Richtárik FedML 19 23 0 02 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 24 14 0 01 Nov 2021
Federated Semi-Supervised Learning with Class Distribution Mismatch Zhiguo Wang Xintong Wang Ruoyu Sun Tsung-Hui Chang FedML 14 12 0 29 Oct 2021
Communication-Efficient ADMM-based Federated Learning Shenglong Zhou Geoffrey Ye Li FedML 35 22 0 28 Oct 2021
Multilayer Lookahead: a Nested Version of Lookahead Denys Pushkin Luis Barba 79 1 0 27 Oct 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 41 71 0 27 Oct 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 39 83 0 26 Oct 2021
TESSERACT: Gradient Flip Score to Secure Federated Learning Against Model Poisoning Attacks Atul Sharma Wei Chen Joshua C. Zhao Qiang Qiu Somali Chaterji S. Bagchi FedML AAML 46 5 0 19 Oct 2021
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 30 19 0 15 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 26 15 0 07 Oct 2021
Paving the Way for Distributed Artificial Intelligence over the Air Guoqing Ma Shuping Dang Chuanting Zhang B. Shihada 11 3 0 24 Sep 2021
Toward Efficient Federated Learning in Multi-Channeled Mobile Edge Network with Layerd Gradient Compression Haizhou Du Xiaojie Feng Qiao Xiang Haoyu Liu 23 0 0 18 Sep 2021
Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer Yae Jee Cho Jianyu Wang Tarun Chiruvolu Gauri Joshi FedML 35 30 0 16 Sep 2021
Federated Submodel Optimization for Hot and Cold Data Features Yucheng Ding Chaoyue Niu Fan Wu Shaojie Tang Chengfei Lv Yanghe Feng Guihai Chen FedML 16 5 0 16 Sep 2021
Exploiting Heterogeneity in Robust Federated Best-Arm Identification A. Mitra Hamed Hassani George Pappas FedML 31 26 0 13 Sep 2021
Cost-Effective Federated Learning in Mobile Edge Networks Bing Luo Xiang Li Shiqiang Wang Jianwei Huang Leandros Tassiulas FedML 51 72 0 12 Sep 2021
Toward Communication Efficient Adaptive Gradient Method Xiangyi Chen Xiaoyun Li P. Li FedML 34 41 0 10 Sep 2021
Byzantine-Robust Federated Learning via Credibility Assessment on Non-IID Data Kun Zhai Qiang Ren Junli Wang Chungang Yan 13 11 0 06 Sep 2021
On Second-order Optimization Methods for Federated Learning Sebastian Bischoff Stephan Günnemann Martin Jaggi Sebastian U. Stich FedML 21 10 0 06 Sep 2021
Communication Efficient Generalized Tensor Factorization for Decentralized Healthcare Networks Jing Ma Qiuchen Zhang Jian Lou Li Xiong S. Bhavani Joyce C. Ho 18 0 0 03 Sep 2021
Statistical Estimation and Inference via Local SGD in Federated Learning Xiang Li Jiadong Liang Xiangyu Chang Zhihua Zhang FedML 22 4 0 03 Sep 2021
Federated Multi-Task Learning under a Mixture of Distributions Othmane Marfoq Giovanni Neglia A. Bellet Laetitia Kameni Richard Vidal FedML 40 269 0 23 Aug 2021
Anarchic Federated Learning Haibo Yang Xin Zhang Prashant Khanduri Jia Liu FedML 16 58 0 23 Aug 2021
Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning Anirban Das Timothy Castiglia Shiqiang Wang S. Patterson FedML 13 19 0 19 Aug 2021
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning Charlie Hou K. K. Thekumparampil Giulia Fanti Sewoong Oh FedML 30 14 0 16 Aug 2021
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning Haoyu Zhao Zhize Li Peter Richtárik FedML 22 29 0 10 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 43 54 0 02 Aug 2021
Communication-Efficient Federated Learning via Predictive Coding Kai Yue Richeng Jin Chau-Wai Wong H. Dai FedML 25 14 0 02 Aug 2021
A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee Chunjiang Che Xiaoli Li Chuan Chen Xiaoyu He Zibin Zheng FedML 31 72 0 01 Aug 2021
DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning Guangfeng Yan Shao-Lun Huang Tian-Shing Lan Linqi Song MQ 6 6 0 30 Jul 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 19 106 0 26 Jul 2021
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time Yuyang Deng Mohammad Mahdi Kamani M. Mahdavi FedML 11 14 0 22 Jul 2021
CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression Zhize Li Peter Richtárik 25 29 0 20 Jul 2021
RingFed: Reducing Communication Costs in Federated Learning on Non-IID Data Guang Yang Ke Mu Chunhe Song Zhijia Yang Tierui Gong FedML 10 16 0 19 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 411 0 14 Jul 2021
FedFog: Network-Aware Optimization of Federated Learning over Wireless Fog-Cloud Systems Van-Dinh Nguyen Symeon Chatzinotas Björn E. Ottersten T. Duong FedML 16 16 0 04 Jul 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 23 30 0 03 Jul 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 16 21 0 02 Jul 2021
On Bridging Generic and Personalized Federated Learning for Image Classification Hong-You Chen Wei-Lun Chao FedML 22 21 0 02 Jul 2021
Personalized Federated Learning with Gaussian Processes Idan Achituve Aviv Shamsian Aviv Navon Gal Chechik Ethan Fetaya FedML 29 98 0 29 Jun 2021
A Non-parametric View of FedAvg and FedProx: Beyond Stationary Points Lili Su Jiaming Xu Pengkun Yang FedML 14 13 0 29 Jun 2021
Federated Noisy Client Learning Huazhu Fu Li Li Bo Han Chengzhong Xu Ling Shao FedML 23 26 0 24 Jun 2021
Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning Hua Huang Fanhua Shang Yuanyuan Liu Hongying Liu FedML 19 14 0 23 Jun 2021
FedCM: Federated Learning with Client-level Momentum Jing Xu Sen Wang Liwei Wang Andrew Chi-Chih Yao FedML 22 94 0 21 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 11 5 0 21 Jun 2021
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning Prashant Khanduri Pranay Sharma Haibo Yang Min-Fong Hong Jia Liu K. Rajawat P. Varshney FedML 19 63 0 19 Jun 2021
Distributed Deep Learning in Open Collaborations Michael Diskin Alexey Bukhtiyarov Max Ryabinin Lucile Saulnier Quentin Lhoest ... Denis Mazur Ilia Kobelev Yacine Jernite Thomas Wolf Gennady Pekhimenko FedML 33 54 0 18 Jun 2021
Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimization Luofeng Liao Li Shen Jia Duan Mladen Kolar Dacheng Tao 27 4 0 18 Jun 2021