RelaySum for Decentralized Deep Learning on Heterogeneous Data

8 October 2021

Thijs Vogels

Lie He

Anastasia Koloskova

Tao R. Lin

Sai Praneeth Karimireddy

Papers citing "RelaySum for Decentralized Deep Learning on Heterogeneous Data"

22 / 22 papers shown

Title
Topology-Aware Knowledge Propagation in Decentralized Learning Mansi Sakarvadia Nathaniel Hudson Tian Li Ian Foster Kyle Chard 22 0 0 16 May 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 67 0 0 31 Jan 2025
Brain-Inspired Decentralized Satellite Learning in Space Computing Power Networks Peng Yang Ting Wang Haibin Cai Yuanming Shi Chunxiao Jiang Linling Kuang 49 0 0 28 Jan 2025
Scalable Decentralized Learning with Teleportation Yuki Takezawa Sebastian U. Stich 72 1 0 25 Jan 2025
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 52 1 0 11 Apr 2024
Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data Sai Aparna Aketi Sakshi Choudhary Kaushik Roy 34 1 0 05 Mar 2024
Topology-Dependent Privacy Bound For Decentralized Federated Learning Qiongxiu Li Wenrui Yu Changlong Ji Richard Heusdens 37 3 0 13 Dec 2023
Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization Zhenwei Lin Jingfan Xia Qi Deng Luo Luo 36 3 0 18 Oct 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 44 9 0 19 May 2023
Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning Deepak Ravikumar Gobinda Saha Sai Aparna Aketi Kaushik Roy 21 2 0 09 Apr 2023
Beyond spectral gap (extended): The role of the topology in decentralized learning Thijs Vogels Hadrien Hendrikx Martin Jaggi 31 3 0 05 Jan 2023
Addressing Data Heterogeneity in Decentralized Learning via Topological Pre-processing Waqwoya Abebe Ali Jannesari 35 0 0 16 Dec 2022
On the effectiveness of partial variance reduction in federated learning with heterogeneous data Bo Li Mikkel N. Schmidt T. S. Alstrøm Sebastian U. Stich FedML 37 9 0 05 Dec 2022
Secure Distributed Optimization Under Gradient Attacks Shuhua Yu S. Kar 39 14 0 28 Oct 2022
Beyond spectral gap: The role of the topology in decentralized learning Thijs Vogels Hadrien Hendrikx Martin Jaggi FedML 21 28 0 07 Jun 2022
Semi-Decentralized Federated Learning with Collaborative Relaying M. Yemini R. Saha Emre Ozfatura Deniz Gündüz Andrea J. Goldsmith FedML 49 32 0 23 May 2022
Data-heterogeneity-aware Mixing for Decentralized Learning Yatin Dandi Anastasia Koloskova Martin Jaggi Sebastian U. Stich 43 18 0 13 Apr 2022
Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying M. Yemini R. Saha Emre Ozfatura Deniz Gündüz Andrea J. Goldsmith FedML 55 8 0 24 Feb 2022
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! Konstantin Mishchenko Grigory Malinovsky Sebastian U. Stich Peter Richtárik 16 152 0 18 Feb 2022
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning Anastasia Koloskova Tao R. Lin Sebastian U. Stich 22 106 0 08 Feb 2022
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning Bicheng Ying Kun Yuan Hanbin Hu Yiming Chen W. Yin FedML 44 27 0 08 Nov 2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training Kun Yuan Yiming Chen Xinmeng Huang Yingya Zhang Pan Pan Yinghui Xu W. Yin MoE 60 61 0 24 Apr 2021