A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

23 March 2020

Papers citing "A Unified Theory of Decentralized SGD with Changing Topology and Local Updates"

50 / 51 papers shown

Title
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 63 0 0 25 Apr 2025
Communication Optimization for Decentralized Learning atop Bandwidth-limited Edge Networks Tingyang Sun Tuan Nguyen Ting He 33 0 0 16 Apr 2025
Decentralized Federated Domain Generalization with Style Sharing: A Formal Modeling and Convergence Analysis Shahryar Zehtabi Dong-Jun Han Seyyedali Hosseinalipour Christopher G. Brinton FedML AI4CE 43 0 0 08 Apr 2025
From Interpretation to Correction: A Decentralized Optimization Framework for Exact Convergence in Federated Learning Bicheng Ying Zhe Li Haibo Yang FedML 68 0 0 25 Mar 2025
Scalable Decentralized Algorithms for Online Personalized Mean Estimation Franco Galante Giovanni Neglia Emilio Leonardi FedML 85 1 0 20 Feb 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 57 0 0 31 Jan 2025
Scalable Decentralized Learning with Teleportation Yuki Takezawa Sebastian U. Stich 54 1 0 25 Jan 2025
Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimization Dun Zeng Zheshun Wu Shiyu Liu Yu Pan Xiaoying Tang Zenglin Xu MLT FedML 71 1 0 25 Nov 2024
OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancement Qinglun Li Miao Zhang Mengzhu Wang Quanjun Yin Li Shen OODD FedML 14 0 0 09 Oct 2024
Peer-to-Peer Learning Dynamics of Wide Neural Networks Shreyas Chaudhari Srinivasa Pranav Emile Anand José M. F. Moura 35 3 0 23 Sep 2024
On the Convergence of a Federated Expectation-Maximization Algorithm Zhixu Tao Rajita Chandak Sanjeev R. Kulkarni FedML 25 0 0 11 Aug 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 45 4 0 06 Jun 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 39 6 0 19 May 2024
Robust Decentralized Learning with Local Updates and Gradient Tracking Sajjad Ghiasvand Amirhossein Reisizadeh Mahnoosh Alizadeh Ramtin Pedarsani 28 3 0 02 May 2024
Impact of network topology on the performance of Decentralized Federated Learning Luigi Palmieri C. Boldrini Lorenzo Valerio A. Passarella M. Conti 19 5 0 28 Feb 2024
FedImpro: Measuring and Improving Client Update in Federated Learning Zhenheng Tang Yonggang Zhang S. Shi Xinmei Tian Tongliang Liu Bo Han Xiaowen Chu FedML 13 13 0 10 Feb 2024
Convergence Analysis of Decentralized ASGD Mauro Dalle Lucca Tosi Martin Theobald 21 2 0 07 Sep 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 15 4 0 21 Jun 2023
Get More for Less in Decentralized Learning Systems Akash Dhasade Anne-Marie Kermarrec Rafael Pires Rishi Sharma Milos Vujasinovic Jeffrey Wigger 13 7 0 07 Jun 2023
Collaborative Learning via Prediction Consensus Dongyang Fan Celestine Mendler-Dünner Martin Jaggi FedML 20 7 0 29 May 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 24 9 0 19 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 30 2 0 09 Apr 2023
Decentralized Learning Made Practical with Client Sampling M. Vos Akash Dhasade Anne-Marie Kermarrec Erick Lavoie J. Pouwelse Rishi Sharma 11 1 0 27 Feb 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 21 10 0 15 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 14 5 0 30 Jan 2023
Entropy-driven Fair and Effective Federated Learning Lung-Chuang Wang Zhichao Wang Sai Praneeth Karimireddy Xiaoying Tang Xiaoying Tang FedML 25 9 0 29 Jan 2023
On the Performance of Gradient Tracking with Local Updates Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe 22 18 0 10 Oct 2022
Byzantine-Resilient Decentralized Stochastic Optimization with Robust Aggregation Rules Zhaoxian Wu Tianyi Chen Qing Ling 18 36 0 09 Jun 2022
A principled framework for the design and analysis of token algorithms Hadrien Hendrikx FedML 14 13 0 30 May 2022
DELTA: Diverse Client Sampling for Fasting Federated Learning Lung-Chuang Wang Yongxin Guo Tao R. Lin Xiaoying Tang FedML 16 20 0 27 May 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 19 74 0 27 May 2022
Federated Learning with Partial Model Personalization Krishna Pillutla Kshitiz Malik Abdel-rahman Mohamed Michael G. Rabbat Maziar Sanjabi Lin Xiao FedML 14 152 0 08 Apr 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 19 46 0 09 Mar 2022
Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying M. Yemini R. Saha Emre Ozfatura Deniz Gündüz Andrea J. Goldsmith FedML 17 8 0 24 Feb 2022
Towards Federated Learning on Time-Evolving Heterogeneous Data Yongxin Guo Tao R. Lin Xiaoying Tang FedML 6 30 0 25 Dec 2021
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 26 414 0 24 Nov 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 15 71 0 27 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 16 15 0 07 Oct 2021
Federated Multi-Task Learning under a Mixture of Distributions Othmane Marfoq Giovanni Neglia A. Bellet Laetitia Kameni Richard Vidal FedML 22 269 0 23 Aug 2021
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning Charlie Hou K. K. Thekumparampil Giulia Fanti Sewoong Oh FedML 14 14 0 16 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 173 410 0 14 Jul 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 11 21 0 02 Jul 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 55 157 0 14 Feb 2021
Privacy Amplification by Decentralization Edwige Cyffers A. Bellet FedML 40 39 0 09 Dec 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 14 1,266 0 15 Jul 2020
Stochastic Hamiltonian Gradient Methods for Smooth Games Nicolas Loizou Hugo Berard Alexia Jolicoeur-Martineau Pascal Vincent Simon Lacoste-Julien Ioannis Mitliagkas 17 51 0 08 Jul 2020
Optimal Complexity in Decentralized Training Yucheng Lu Christopher De Sa 8 71 0 15 Jun 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 11 198 0 08 Jun 2020
New Convergence Aspects of Stochastic Gradient Algorithms Lam M. Nguyen Phuong Ha Nguyen Peter Richtárik K. Scheinberg Martin Takáč Marten van Dijk 18 65 0 10 Nov 2018
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method Simon Lacoste-Julien Mark W. Schmidt Francis R. Bach 111 259 0 10 Dec 2012