Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation Farshid Varno Marzie Saghayi Laya Rafiee Sharut Gupta Stan Matwin Mohammad Havaei FedML 34 30 0 27 Apr 2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning Samuel Horváth Maziar Sanjabi Lin Xiao Peter Richtárik Michael G. Rabbat FedML 25 21 0 27 Apr 2022
Federated Learning via Inexact ADMM Shenglong Zhou Geoffrey Ye Li FedML 24 58 0 22 Apr 2022
FedCorr: Multi-Stage Federated Learning for Label Noise Correction Jingyi Xu Zihan Chen Tony Q. S. Quek Kai Fong Ernest Chong FedML 16 85 0 10 Apr 2022
FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity Yonghai Gong Yichuan Li N. Freris FedML 14 45 0 07 Apr 2022
FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participation Han Wang Siddartha Marella James Anderson FedML 6 39 0 28 Mar 2022
SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networks Won Joon Yun Yunseok Kwak Hankyul Baek Soyi Jung Mingyue Ji M. Bennis Jihong Park Joongheon Kim 18 16 0 26 Mar 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 27 3 0 22 Mar 2022
The Role of Local Steps in Local SGD Tiancheng Qin S. Rasoul Etesami César A. Uribe 13 4 0 14 Mar 2022
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 19 2 0 13 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 21 47 0 09 Mar 2022
BagPipe: Accelerating Deep Recommendation Model Training Saurabh Agarwal Chengpo Yan Ziyi Zhang Shivaram Venkataraman 29 17 0 24 Feb 2022
FedCAT: Towards Accurate Federated Learning via Device Concatenation Ming Hu Tian Liu Zhiwei Ling Zhihao Yue Mingsong Chen FedML 16 1 0 23 Feb 2022
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! Konstantin Mishchenko Grigory Malinovsky Sebastian U. Stich Peter Richtárik 11 149 0 18 Feb 2022
On the Convergence of Clustered Federated Learning Ma Jie Guodong Long Dinesh Manocha Jing Jiang Chengqi Zhang FedML 36 46 0 13 Feb 2022
On Federated Learning with Energy Harvesting Clients Cong Shen Jing Yang Jie Xu FedML 14 6 0 12 Feb 2022
Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning Tomoya Murata Taiji Suzuki FedML 6 2 0 12 Feb 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
Comparative assessment of federated and centralized machine learning Ibrahim Abdul Majeed Sagar Kaushik Aniruddha Bardhan Venkata Siva Kumar Tadi Hwang-Ki Min K. Kumaraguru Rajasekhara Reddy Duvvuru Muni FedML 20 6 0 03 Feb 2022
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation Peter Richtárik Igor Sokolov Ilyas Fatkhullin Elnur Gasanov Zhize Li Eduard A. Gorbunov 15 31 0 02 Feb 2022
Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank? Sheikh Shams Azam Seyyedali Hosseinalipour Qiang Qiu Christopher G. Brinton FedML 20 20 0 01 Feb 2022
Faster Convergence of Local SGD for Over-Parameterized Models Tiancheng Qin S. Rasoul Etesami César A. Uribe FedML 33 6 0 30 Jan 2022
Gradient Masked Averaging for Federated Learning Irene Tenison Sai Aravind Sreeramadas Vaikkunth Mugunthan Edouard Oyallon Irina Rish Eugene Belilovsky FedML 68 24 0 28 Jan 2022
On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning Hanhan Zhou Tian-Shing Lan Guru Venkataramani Wenbo Ding FedML 24 6 0 27 Jan 2022
Achieving Personalized Federated Learning with Sparse Local Models Tiansheng Huang Shiwei Liu Li Shen Fengxiang He Weiwei Lin Dacheng Tao FedML 30 43 0 27 Jan 2022
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks Or Litany Haggai Maron David Acuna Jan Kautz Gal Chechik Sanja Fidler FedML 38 24 0 20 Jan 2022
Variance-Reduced Heterogeneous Federated Learning via Stratified Client Selection Guangyuan Shen D. Gao Libin Yang Fang Zhou Duanxiao Song Wei Lou Shirui Pan FedML 17 8 0 15 Jan 2022
Partial Model Averaging in Federated Learning: Performance Guarantees and Benefits Sunwoo Lee Anit Kumar Sahu Chaoyang He Salman Avestimehr FedML 25 16 0 11 Jan 2022
Communication-Efficient Federated Learning with Accelerated Client Gradient Geeho Kim Jinkyu Kim Bohyung Han FedML 37 11 0 10 Jan 2022
AET-SGD: Asynchronous Event-triggered Stochastic Gradient Descent Nhuong V. Nguyen Song Han 21 2 0 27 Dec 2021
Automatic Configuration for Optimal Communication Scheduling in DNN Training Yiqing Ma Hao Wang Yiming Zhang Kai Chen 17 12 0 27 Dec 2021
Towards Federated Learning on Time-Evolving Heterogeneous Data Yongxin Guo Tao R. Lin Xiaoying Tang FedML 22 30 0 25 Dec 2021
Sparsified Secure Aggregation for Privacy-Preserving Federated Learning Irem Ergun Hasin Us Sami Başak Güler FedML 28 25 0 23 Dec 2021
FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 30 25 0 22 Dec 2021
Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling Bing Luo Wenli Xiao Shiqiang Wang Jianwei Huang Leandros Tassiulas FedML 34 168 0 21 Dec 2021
From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization Feijie Wu Song Guo Yining Qi Zhihao Qu Haobo Zhang Jiewei Zhang Ziming Liu 25 11 0 17 Dec 2021
LoSAC: An Efficient Local Stochastic Average Control Method for Federated Optimization Huiming Chen Huandong Wang Quanming Yao Yong Li Depeng Jin Qiang Yang FedML 11 4 0 15 Dec 2021
Communication-Efficient Distributed SGD with Compressed Sensing Yujie Tang V. Ramanathan Junshan Zhang Na Li FedML 12 8 0 15 Dec 2021
SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification Ashwinee Panda Saeed Mahloujifar A. Bhagoji Supriyo Chakraborty Prateek Mittal FedML AAML 9 84 0 12 Dec 2021
On Convergence of Federated Averaging Langevin Dynamics Wei Deng Qian Zhang Yi Ma Zhao-quan Song Guang Lin FedML 27 16 0 09 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 22 11 0 07 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 38 16 0 05 Dec 2021
Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks Hankyul Baek Won Joon Yun Yunseok Kwak Soyi Jung Mingyue Ji M. Bennis Jihong Park Joongheon Kim FedML 71 21 0 05 Dec 2021
Learn Locally, Correct Globally: A Distributed Algorithm for Training Graph Neural Networks M. Ramezani Weilin Cong Mehrdad Mahdavi M. Kandemir A. Sivasubramaniam GNN 20 32 0 16 Nov 2021
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters Xiangru Lian Binhang Yuan Xuefeng Zhu Yulong Wang Yongjun He ... Lei Yuan Hai-bo Yu Sen Yang Ce Zhang Ji Liu VLM 25 34 0 10 Nov 2021
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning Bicheng Ying Kun Yuan Hanbin Hu Yiming Chen W. Yin FedML 31 27 0 08 Nov 2021
Federated Learning Based on Dynamic Regularization D. A. E. Acar Yue Zhao Ramon Matas Navarro Matthew Mattina P. Whatmough Venkatesh Saligrama FedML 13 746 0 08 Nov 2021
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Margalit Glasgow Honglin Yuan Tengyu Ma FedML 27 43 0 05 Nov 2021
Data Selection for Efficient Model Update in Federated Learning Hongrui Shi Valentin Radu FedML 29 5 0 05 Nov 2021