Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 202 papers shown

Title
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 26 0 0 01 May 2025
BackSlash: Rate Constrained Optimized Training of Large Language Models Jun Wu Jiangtao Wen Yuxing Han 34 0 0 23 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 41 0 0 08 Apr 2025
Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning Haoran Zhang Zejun Gong Zekai Li Marie Siew Carlee Joe-Wong Rachid El-Azouzi 31 0 0 07 Apr 2025
FedSAUC: A Similarity-Aware Update Control for Communication-Efficient Federated Learning in Edge Computing Ming-Lun Lee Han-Chang Chou Yan-AnnChen FedML 34 6 0 07 Apr 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 43 0 0 11 Mar 2025
Delayed Random Partial Gradient Averaging for Federated Learning Xinyi Hu FedML 43 0 0 31 Dec 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 46 1 0 28 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 31 2 0 21 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 33 0 0 04 Oct 2024
Novel Gradient Sparsification Algorithm via Bayesian Inference Ali Bereyhi B. Liang G. Boudreau Ali Afana 34 2 0 23 Sep 2024
Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework Zilinghan Li Shilan He Ze Yang Minseok Ryu Kibaek Kim Ravi K. Madduri FedML 52 5 0 17 Sep 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 36 0 0 13 Sep 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 52 0 0 27 Jul 2024
Asynchronous Byzantine Federated Learning Bart Cox Abele Malan Lydia Y. Chen Jérémie Decouchant 42 1 0 03 Jun 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections Roy Miles Pradyumna Reddy Ismail Elezi Jiankang Deng VLM 32 3 0 28 May 2024
Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Scheme Qianyu Long Qiyuan Wang Christos Anagnostopoulos Daning Bi FedML 26 0 0 24 Apr 2024
Federated Multi-Agent Mapping for Planetary Exploration Tiberiu-Ioan Szatmari Abhishek Cauligi FedML AI4CE 37 0 0 02 Apr 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 38 1 0 25 Mar 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes Xiaoxin Su Yipeng Zhou Laizhong Cui John C. S. Lui Jiangchuan Liu FedML 27 1 0 06 Feb 2024
Correlated Quantization for Faster Nonconvex Distributed Optimization Andrei Panferov Yury Demidovich Ahmad Rammal Peter Richtárik MQ 28 4 0 10 Jan 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Federated Continual Learning via Knowledge Fusion: A Survey Xin Yang Hao Yu Xin Gao Hao Wang Junbo Zhang Tianrui Li FedML 30 31 0 27 Dec 2023
Compressed and Sparse Models for Non-Convex Decentralized Learning Andrew Campbell Hang Liu Leah Woldemariam Anna Scaglione 20 0 0 09 Nov 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 18 1 0 24 Oct 2023
FedDIP: Federated Learning with Extreme Dynamic Pruning and Incremental Regularization Qianyu Long Christos Anagnostopoulos S. P. Parambath Daning Bi AI4CE FedML 11 2 0 13 Sep 2023
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 70 4 0 01 Aug 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 28 0 0 01 Aug 2023
Mitigating Cross-client GANs-based Attack in Federated Learning Hong Huang Xinyu Lei Tao Xiang AAML 45 1 0 25 Jul 2023
Accelerating Distributed ML Training via Selective Synchronization S. Tyagi Martin Swany FedML 24 3 0 16 Jul 2023
FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout Jingjing Xue Min Liu Sheng Sun Yuwei Wang Hui Jiang Xue Jiang 18 7 0 14 Jul 2023
Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation Berivan Isik Wei-Ning Chen Ayfer Özgür Tsachy Weissman Albert No 48 19 0 08 Jun 2023
Get More for Less in Decentralized Learning Systems Akash Dhasade Anne-Marie Kermarrec Rafael Pires Rishi Sharma Milos Vujasinovic Jeffrey Wigger 26 7 0 07 Jun 2023
Theoretically Principled Federated Learning for Balancing Privacy and Utility Xiaojin Zhang Wenjie Li Kai Chen Shutao Xia Qian Yang FedML 17 9 0 24 May 2023
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Improving Small Language Models on PubMedQA via Generative Data Augmentation Zhen Guo Peiqi Wang Yanwei Wang Shangdi Yu LM&MA MedIm 18 10 0 12 May 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 30 4 0 15 Apr 2023
Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy Zhenxiao Zhang Yuanxiong Guo Yuguang Fang Yanmin Gong 28 4 0 15 Apr 2023
RecUP-FL: Reconciling Utility and Privacy in Federated Learning via User-configurable Privacy Defense Yue-li Cui Syed Imran Ali Meerza Zhuohang Li Luyang Liu Jiaxin Zhang Jian-Dong Liu AAML FedML 21 4 0 11 Apr 2023
FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems Rui Song Runsheng Xu Andreas Festag Jiaqi Ma Alois C. Knoll FedML 28 25 0 04 Apr 2023
Complement Sparsification: Low-Overhead Model Pruning for Federated Learning Xiaopeng Jiang Cristian Borcea FedML 26 15 0 10 Mar 2023
Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training W. Tan Xiao Shi Cunchi Lv Xiaofang Zhao FedML 20 1 0 09 Mar 2023
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment Shiwei Zhang Xiaodong Yi Lansong Diao Chuan Wu Siyu Wang W. Lin GNN 11 5 0 13 Feb 2023
FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation Hanlin Gu Jiahuan Luo Yan Kang Lixin Fan Qiang Yang FedML 34 13 0 30 Jan 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 22 31 0 27 Jan 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 14 25 0 24 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 11 4 0 23 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 19 4 0 06 Jan 2023
Mutual Information Regularization for Vertical Federated Learning Tianyuan Zou Yang Liu Ya-Qin Zhang AAML FedML 24 7 0 01 Jan 2023