Stochastic Gradient Push for Distributed Deep Learning

27 November 2018

Papers citing "Stochastic Gradient Push for Distributed Deep Learning"

50 / 54 papers shown

Title
Dyn-D $^2$ P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee Z. Zhu Y. Huang Xin Wang Shouling Ji Jinming Xu 26 0 0 10 May 2025
Hubs and Spokes Learning: Efficient and Scalable Collaborative Machine Learning Atul Sharma Kavindu Herath Saurabh Bagchi Chaoyue Liu Somali Chaterji FedML 42 0 0 29 Apr 2025
From Interpretation to Correction: A Decentralized Optimization Framework for Exact Convergence in Federated Learning Bicheng Ying Zhe Li Haibo Yang FedML 68 0 0 25 Mar 2025
Scalable Decentralized Learning with Teleportation Yuki Takezawa Sebastian U. Stich 56 1 0 25 Jan 2025
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 64 0 0 26 Nov 2024
Fair Decentralized Learning Sayan Biswas Anne-Marie Kermarrec Rishi Sharma Thibaud Trinca M. Vos FedML 32 0 0 03 Oct 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 53 4 0 06 Jun 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 42 0 0 22 May 2024
Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors Chun-Yin Huang Kartik Srinivas Xin Zhang Xiaoxiao Li DD 50 6 0 19 May 2024
PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Bounds Zehan Zhu Yan Huang Xin Wang Jinming Xu 37 0 0 04 May 2024
Sequence-to-sequence models in peer-to-peer learning: A practical application Robert Šajina Ivo Ipšić 38 0 0 02 May 2024
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 28 2 0 18 Jun 2023
Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates Siqi Zhang S. Choudhury Sebastian U. Stich Nicolas Loizou FedML 19 3 0 08 Jun 2023
Collaborative Learning via Prediction Consensus Dongyang Fan Celestine Mendler-Dünner Martin Jaggi FedML 29 7 0 29 May 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 31 9 0 19 May 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression Boyue Li Yuejie Chi 21 12 0 17 May 2023
Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning Deepak Ravikumar Gobinda Saha Sai Aparna Aketi Kaushik Roy 11 1 0 09 Apr 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 45 3 0 27 Mar 2023
Beyond spectral gap (extended): The role of the topology in decentralized learning Thijs Vogels Hadrien Hendrikx Martin Jaggi 22 3 0 05 Jan 2023
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 27 10 0 15 Sep 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 26 75 0 27 May 2022
Semi-Decentralized Federated Learning with Collaborative Relaying M. Yemini R. Saha Emre Ozfatura Deniz Gündüz Andrea J. Goldsmith FedML 34 32 0 23 May 2022
CFL: Cluster Federated Learning in Large-scale Peer-to-Peer Networks Qian Chen Zilong Wang Yilin Zhou Jiawei Chen Danyang Xiao Xiaodong Lin FedML 17 6 0 08 Apr 2022
Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying M. Yemini R. Saha Emre Ozfatura Deniz Gündüz Andrea J. Goldsmith FedML 40 8 0 24 Feb 2022
Variance-Reduced Stochastic Quasi-Newton Methods for Decentralized Learning: Part I Jiaojiao Zhang Huikang Liu Anthony Man-Cho So Qing Ling 11 14 0 19 Jan 2022
Distributed Random Reshuffling over Networks Kun-Yen Huang Xiao Li Andre Milzarek Shi Pu Junwen Qiu 36 11 0 31 Dec 2021
Personalized On-Device E-health Analytics with Decentralized Block Coordinate Descent Guanhua Ye Hongzhi Yin Tong Chen Miao Xu Quoc Viet Hung Nguyen Jiangning Song 37 9 0 17 Dec 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 36 83 0 26 Oct 2021
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning Sulaiman A. Alghunaim Kun Yuan 25 57 0 19 Oct 2021
On the Convergence of Decentralized Adaptive Gradient Methods Xiangyi Chen Belhal Karimi Weijie Zhao Ping Li 19 21 0 07 Sep 2021
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines Shigang Li Torsten Hoefler GNN AI4CE LRM 77 131 0 14 Jul 2021
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities Aleksandr Beznosikov Pavel Dvurechensky Anastasia Koloskova V. Samokhin Sebastian U. Stich Alexander Gasnikov 26 43 0 15 Jun 2021
From Distributed Machine Learning to Federated Learning: A Survey Ji Liu Jizhou Huang Yang Zhou Xuhong Li Shilei Ji Haoyi Xiong Dejing Dou FedML OOD 44 243 0 29 Apr 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 29 31 0 04 Mar 2021
Sparse-Push: Communication- & Energy-Efficient Decentralized Distributed Learning over Directed & Time-Varying Graphs with non-IID Datasets Sai Aparna Aketi Amandeep Singh J. Rabaey 21 10 0 10 Feb 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications Shuyan Hu Xiaojing Chen Wei Ni E. Hossain Xin Wang AI4CE 42 111 0 02 Dec 2020
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery Kiwan Maeng Shivam Bharuka Isabel Gao M. C. Jeffrey V. Saraph ... Caroline Trippel Jiyan Yang Michael G. Rabbat Brandon Lucia Carole-Jean Wu OffRL 11 31 0 05 Nov 2020
FederBoost: Private Federated Learning for GBDT Zhihua Tian Rui Zhang Xiaoyang Hou Jian-wei Liu K. Ren Jian Liu Kui Ren FedML AI4CE 39 66 0 05 Nov 2020
Throughput-Optimal Topology Design for Cross-Silo Federated Learning Othmane Marfoq Chuan Xu Giovanni Neglia Richard Vidal FedML 64 85 0 23 Oct 2020
Stochastic Hamiltonian Gradient Methods for Smooth Games Nicolas Loizou Hugo Berard Alexia Jolicoeur-Martineau Pascal Vincent Simon Lacoste-Julien Ioannis Mitliagkas 28 50 0 08 Jul 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 21 12 0 07 Jul 2020
Optimal Complexity in Decentralized Training Yucheng Lu Christopher De Sa 25 71 0 15 Jun 2020
S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs Muhammad I. Qureshi Ran Xin S. Kar U. Khan 15 33 0 15 May 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning Yucheng Lu J. Nash Christopher De Sa FedML 21 12 0 14 May 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 36 9 0 11 Apr 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 39 491 0 23 Mar 2020
Decentralized gradient methods: does topology matter? Giovanni Neglia Chuan Xu Don Towsley G. Calbi 11 50 0 28 Feb 2020
Gradient tracking and variance reduction for decentralized optimization and machine learning Ran Xin S. Kar U. Khan 16 10 0 13 Feb 2020
Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning XINYAN DAI Xiao Yan Kaiwen Zhou Han Yang K. K. Ng James Cheng Yu Fan FedML 6 47 0 12 Nov 2019