Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

9 February 2021

Tao R. Lin

Sai Praneeth Karimireddy

Papers citing "Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data"

28 / 28 papers shown

Title
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution Lina Wang Yunsheng Yuan Chunxiao Wang Feng Li FedML 43 0 0 31 Mar 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 57 0 0 31 Jan 2025
Scalable Decentralized Learning with Teleportation Yuki Takezawa Sebastian U. Stich 56 1 0 25 Jan 2025
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 54 0 0 27 Jul 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 42 0 0 22 May 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 42 1 0 11 Apr 2024
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions Siyuan Zhang Nachuan Xiao Xin Liu 61 1 0 18 Mar 2024
Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data Sai Aparna Aketi Sakshi Choudhary Kaushik Roy 30 1 0 05 Mar 2024
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 19 16 0 30 Jul 2023
Achieving Linear Speedup in Decentralized Stochastic Compositional Minimax Optimization Hongchang Gao 30 1 0 25 Jul 2023
Structured Cooperative Learning with Graphical Model Priors Shuang-Yang Li Tianyi Zhou Xinmei Tian Dacheng Tao 30 0 0 16 Jun 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 31 9 0 19 May 2023
Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning Deepak Ravikumar Gobinda Saha Sai Aparna Aketi Kaushik Roy 11 1 0 09 Apr 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 45 3 0 27 Mar 2023
Beyond spectral gap (extended): The role of the topology in decentralized learning Thijs Vogels Hadrien Hendrikx Martin Jaggi 22 3 0 05 Jan 2023
Aggregation in the Mirror Space (AIMS): Fast, Accurate Distributed Machine Learning in Military Settings Ryan Yang Haizhou Du Andre Wibisono Patrick Baker 11 1 0 28 Oct 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 27 10 0 15 Sep 2022
Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching Zexi Li Jiaxun Lu Shuang Luo Didi Zhu Yunfeng Shao Yinchuan Li Zhimeng Zhang Yongheng Wang Chao-Xiang Wu FedML 30 36 0 23 Mar 2022
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 34 416 0 24 Nov 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 36 83 0 26 Oct 2021
FedCM: Federated Learning with Client-level Momentum Jing Xu Sen Wang Liwei Wang Andrew Chi-Chih Yao FedML 22 94 0 21 Jun 2021
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities Aleksandr Beznosikov Pavel Dvurechensky Anastasia Koloskova V. Samokhin Sebastian U. Stich Alexander Gasnikov 26 43 0 15 Jun 2021
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization Xiaoxiao Li Meirui Jiang Xiaofei Zhang Michael Kamp Qi Dou OOD FedML 168 787 0 15 Feb 2021
Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training Shen-Yi Zhao Chang-Wei Shi Yin-Peng Xie Wu-Jun Li ODL 18 8 0 28 Jul 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 168 564 0 27 Jul 2020
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 221 1,399 0 04 Dec 2018
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 78 129 0 16 Oct 2018
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 171 683 0 07 Dec 2010