Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
Energy-Efficient Federated Learning for AIoT using Clustering Methods Roberto Pereira Fernanda Famá Charalampos Kalalas Paolo Dini 14 0 0 14 May 2025
Sharp Gaussian approximations for Decentralized Federated Learning Soham Bonnerjee Sayar Karmakar W. Wu FedML 24 0 0 12 May 2025
Cluster-Aware Multi-Round Update for Wireless Federated Learning in Heterogeneous Environments Pengcheng Sun Erwu Liu Wei Ni Kanglei Yu Rui-cang Wang Abbas Jamalipour FedML 19 0 0 06 May 2025
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 63 0 0 25 Apr 2025
A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip Peyman Gholami H. Seferoglu 24 0 0 14 Apr 2025
Federated Learning for Medical Image Classification: A Comprehensive Benchmark Zhekai Zhou Guibo Luo Mingzhi Chen Zhenyu Weng Yuesheng Zhu FedML 26 0 0 07 Apr 2025
Client Selection in Federated Learning with Data Heterogeneity and Network Latencies Harsh Vardhan Xiaofan Yu Tajana Rosing A. Mazumdar FedML 39 0 0 02 Apr 2025
Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learning Karlo Palenzuela Ali Dadras A. Yurtsever Tommy Löfstedt FedML 40 0 0 27 Mar 2025
From Interpretation to Correction: A Decentralized Optimization Framework for Exact Convergence in Federated Learning Bicheng Ying Zhe Li Haibo Yang FedML 68 0 0 25 Mar 2025
Sense4FL: Vehicular Crowdsensing Enhanced Federated Learning for Autonomous Driving Yanan Ma Senkang Hu Zhengru Fang Yun Ji Yiqin Deng Yuguang Fang 46 0 0 22 Mar 2025
A Flexible Fairness Framework with Surrogate Loss Reweighting for Addressing Sociodemographic Disparities Wen Xu Elham Dolatabadi FaML 89 0 0 21 Mar 2025
Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Framework Erik G. Larsson Nicolo Michelusi 52 0 0 18 Mar 2025
Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo Zachary B. Charles Gabriel Teston Lucio Dery Keith Rush Nova Fallen Zachary Garrett Arthur Szlam Arthur Douillard 140 0 0 12 Mar 2025
Biased Federated Learning under Wireless Heterogeneity Muhammad Faraz Ul Abrar Nicolò Michelusi FedML 44 0 0 08 Mar 2025
Convergence Analysis of Federated Learning Methods Using Backward Error Analysis Jinwoo Lim Suhyun Kim Soo-Mook Moon FedML 53 0 0 05 Mar 2025
Multi-Target Federated Backdoor Attack Based on Feature Aggregation Lingguag Hao K. Hao Bing Wei Xue-song Tang FedML AAML 54 0 0 23 Feb 2025
Preconditioned Inexact Stochastic ADMM for Deep Model Shenglong Zhou Ouya Wang Ziyan Luo Yongxu Zhu Geoffrey Ye Li 41 0 0 15 Feb 2025
Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning Donglin Zhan Leonardo F. Toso James Anderson 97 1 0 04 Feb 2025
Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimization Aditya Devarakonda Ramakrishnan Kannan FedML 37 0 0 13 Jan 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 38 0 0 08 Jan 2025
A Unified Analysis of Federated Learning with Arbitrary Client Participation Shiqiang Wang Mingyue Ji FedML 29 55 0 31 Dec 2024
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 77 1 0 22 Dec 2024
Task Diversity in Bayesian Federated Learning: Simultaneous Processing of Classification and Regression Junliang Lyu Yixuan Zhang Xiaoling Lu Feng Zhou FedML 75 0 0 14 Dec 2024
Review of Mathematical Optimization in Federated Learning Shusen Yang Fangyuan Zhao Zihao Zhou Liang Shi Xuebin Ren Zongben Xu FedML AI4CE 70 0 0 02 Dec 2024
INTELLECT-1 Technical Report Sami Jaghouar Jack Min Ong Manveer Basra Fares Obeid Jannik Straube ... Lucas Atkins Maziyar Panahi Charles Goddard Max Ryabinin Johannes Hagemann MoE 65 1 0 02 Dec 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 183 3 0 20 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Photon: Federated LLM Pre-Training Lorenzo Sani Alex Iacob Zeyu Cao Royson Lee Bill Marino ... Dongqi Cai Zexi Li Wanru Zhao Xinchi Qiu Nicholas D. Lane AI4CE 26 7 0 05 Nov 2024
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training Jinda Jia Cong Xie Hanlin Lu Daoce Wang Hao Feng ... Baixi Sun Haibin Lin Zhi-Li Zhang Xin Liu Dingwen Tao MQ 25 4 0 20 Oct 2024
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications Yue Huang Zhaoxian Wu Shiqian Ma Qing Ling 31 1 0 17 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Mohit Bansal Tianlong Chen MoMe MoE 25 1 0 09 Oct 2024
Communication-Efficient Federated Group Distributionally Robust Optimization Zhishuai Guo Tianbao Yang FedML 28 0 0 08 Oct 2024
Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information Emanuel Buttaci Giuseppe Carlo Calafiore FedML 21 0 0 07 Oct 2024
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 136 0 0 07 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 33 0 0 04 Oct 2024
Collaborative and Efficient Personalization with Mixtures of Adaptors Abdulla Jasem Almansoori Samuel Horváth Martin Takáč FedML 42 2 0 04 Oct 2024
Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients Youssef Allouah Abdellah El Mrini R. Guerraoui Nirupam Gupta Rafael Pinot FedML 27 0 0 30 Sep 2024
Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning Zhidong Gao Zhenxiao Zhang Yu Zhang Tongnian Wang Yanmin Gong Yuanxiong Guo 30 0 0 29 Sep 2024
Hierarchical Federated Learning with Multi-Timescale Gradient Correction Wenzhi Fang Dong-Jun Han Evan Chen Shiqiang Wang Christopher G. Brinton 29 4 0 27 Sep 2024
Does Worst-Performing Agent Lead the Pack? Analyzing Agent Dynamics in Unified Distributed SGD Jie Hu Yi-Ting Ma Do Young Eun FedML 27 0 0 26 Sep 2024
Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability Ming Xiang Stratis Ioannidis Edmund Yeh Carlee Joe-Wong Lili Su FedML 21 4 0 26 Sep 2024
Riemannian Federated Learning via Averaging Gradient Stream Zhenwei Huang Wen Huang Pratik Jawanpuria Bamdev Mishra FedML 24 1 0 11 Sep 2024
SAB:A Stealing and Robust Backdoor Attack based on Steganographic Algorithm against Federated Learning Weida Xu Yang Xu Sicong Zhang AAML FedML 19 0 0 25 Aug 2024
Submodular Maximization Approaches for Equitable Client Selection in Federated Learning Andrés Catalino Castillo Jiménez Ege C. Kaya Lintao Ye Abolfazl Hashemi FedML 49 2 0 24 Aug 2024
Understanding Data Reconstruction Leakage in Federated Learning from a Theoretical Perspective Zifan Wang Binghui Zhang Meng Pang Yuan Hong Binghui Wang FedML 36 0 0 22 Aug 2024
Federated Frank-Wolfe Algorithm Ali Dadras Sourasekhar Banerjee Karthik Prakhya A. Yurtsever FedML 32 4 0 19 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Page-Caccia Haokun Liu Tianlong Chen Mohit Bansal Leshem Choshen Alessandro Sordoni MoMe 41 21 0 13 Aug 2024
Rina: Enhancing Ring-AllReduce with In-network Aggregation in Distributed Model Training Zixuan Chen Xuandong Liu Minglin Li Yinfan Hu Hao Mei Huifeng Xing Hao Wang Wanxin Shi Sen Liu Yang Xu 16 0 0 29 Jul 2024
FADAS: Towards Federated Adaptive Asynchronous Optimization Yujia Wang Shiqiang Wang Songtao Lu Jinghui Chen FedML 34 3 0 25 Jul 2024
Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence Shuya Feng Meisam Mohammady Hanbin Hong Shenao Yan Ashish Kundu Binghui Wang Yuan Hong FedML 36 3 0 20 Jul 2024