Learned Gradient Compression for Distributed Deep Learning

Learned Gradient Compression for Distributed Deep Learning

16 March 2021

Giannis Bekoulis

Nikos Deligiannis

Papers citing "Learned Gradient Compression for Distributed Deep Learning"

16 / 16 papers shown

Title
CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models Huiwen Wu Xiaohan Li Deyi Zhang Xiaogang Xu Jiafei Wu Puning Zhao Zhe Liu FedML 35 0 0 22 May 2024
AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning D. Coquelin Katherina Flügel Marie Weiel Nicholas Kiefer Muhammed Öz Charlotte Debus Achim Streit Markus Goetz 34 0 0 02 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 29 6 0 09 Apr 2024
HSViT: Horizontally Scalable Vision Transformer Chenhao Xu Chang-Tsun Li Chee Peng Lim Douglas Creighton ViT 27 2 0 08 Apr 2024
Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning Daegun Yoon Sangyoon Oh 38 0 0 21 Feb 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Learned layered coding for Successive Refinement in the Wyner-Ziv Problem Boris Joukovsky B. D. Weerdt Nikos Deligiannis 21 2 0 06 Nov 2023
MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training Daegun Yoon Sangyoon Oh 21 1 0 02 Oct 2023
DEFT: Exploiting Gradient Norm Difference between Model Layers for Scalable Gradient Sparsification Daegun Yoon Sangyoon Oh 20 1 0 07 Jul 2023
DISCO: Distributed Inference with Sparse Communications Minghai Qin Chaowen Sun Jaco A. Hofmann D. Vučinić FedML 22 1 0 22 Feb 2023
Encoded Gradients Aggregation against Gradient Leakage in Federated Learning Dun Zeng Shiyu Liu Siqi Liang Zonghang Li Hongya Wang Irwin King Zenglin Xu FedML 6 0 0 26 May 2022
Enabling All In-Edge Deep Learning: A Literature Review Praveen Joshi Mohammed Hasanuzzaman Chandra Thapa Haithem Afli T. Scully 21 22 0 07 Apr 2022
Federated learning and next generation wireless communications: A survey on bidirectional relationship Debaditya Shome Omer Waqar Wali Ullah Khan 26 31 0 14 Oct 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 27 66 0 21 Apr 2021
A Review on Fact Extraction and Verification Giannis Bekoulis Christina Papagiannopoulou Nikos Deligiannis 25 41 0 06 Oct 2020
Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights Aojun Zhou Anbang Yao Yiwen Guo Lin Xu Yurong Chen MQ 311 1,047 0 10 Feb 2017