Is Local SGD Better than Minibatch SGD?

18 February 2020

Papers citing "Is Local SGD Better than Minibatch SGD?"

31 / 31 papers shown

Title
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 16 1 0 22 Jul 2024
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 22 0 0 17 Jun 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 39 6 0 19 May 2024
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 19 16 0 30 Jul 2023
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape Yan Sun Li Shen Shi-Yong Chen Liang Ding Dacheng Tao FedML 19 33 0 19 May 2023
Faster Federated Learning with Decaying Number of Local SGD Steps Jed Mills Jia Hu Geyong Min FedML 20 7 0 16 May 2023
Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling Nicolò Dal Fabbro A. Mitra George J. Pappas FedML 22 12 0 14 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 32 2 0 09 Apr 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 21 10 0 15 Feb 2023
Delay Sensitive Hierarchical Federated Learning with Stochastic Local Updates Abdulmoneam Ali A. Arafa FedML 25 3 0 09 Feb 2023
Learning to Generate Image Embeddings with User-level Differential Privacy Zheng Xu Maxwell D. Collins Yuxiao Wang Liviu Panait Sewoong Oh S. Augenstein Ting Liu Florian Schroff H. B. McMahan FedML 20 29 0 20 Nov 2022
Label driven Knowledge Distillation for Federated Learning with non-IID Data Minh-Duong Nguyen Viet Quoc Pham D. Hoang Long Tran-Thanh Diep N. Nguyen W. Hwang 13 2 0 29 Sep 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 13 1 0 25 Sep 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 19 74 0 27 May 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 17 3 0 22 Mar 2022
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning A. Bietti Chen-Yu Wei Miroslav Dudík John Langford Zhiwei Steven Wu FedML 14 43 0 10 Feb 2022
Towards Federated Learning on Time-Evolving Heterogeneous Data Yongxin Guo Tao R. Lin Xiaoying Tang FedML 6 30 0 25 Dec 2021
Federated Expectation Maximization with heterogeneity mitigation and variance reduction Aymeric Dieuleveut G. Fort Eric Moulines Geneviève Robin FedML 23 5 0 03 Nov 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 16 15 0 07 Oct 2021
Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer Yae Jee Cho Jianyu Wang Tarun Chiruvolu Gauri Joshi FedML 6 29 0 16 Sep 2021
Accelerating Federated Learning with a Global Biased Optimiser Jed Mills Jia Hu Geyong Min Rui Jin Siwei Zheng Jin Wang FedML AI4CE 25 9 0 20 Aug 2021
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning Charlie Hou K. K. Thekumparampil Giulia Fanti Sewoong Oh FedML 19 14 0 16 Aug 2021
Implicit Gradient Alignment in Distributed and Federated Learning Yatin Dandi Luis Barba Martin Jaggi FedML 11 31 0 25 Jun 2021
Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning Zachary B. Charles Jakub Konecný FedML 14 61 0 08 Mar 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 55 157 0 14 Feb 2021
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 6 159 0 06 Aug 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 14 1,293 0 15 Jul 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 13 198 0 08 Jun 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 19 181 0 11 May 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 27 489 0 23 Mar 2020
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 166 683 0 07 Dec 2010