Communication-Efficient Large-Scale Distributed Deep Learning: A
Comprehensive Survey

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey

9 April 2024

Victor C. M. Leung

Xiping Hu

Papers citing "Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey"

15 / 15 papers shown

Title
VAE-based Feature Disentanglement for Data Augmentation and Compression in Generalized GNSS Interference Classification Lucas Heublein Simon Kocher Tobias Feigl A. Rügamer Christopher Mutschler Felix Ott DRL 29 1 0 14 Apr 2025
VcLLM: Video Codecs are Secretly Tensor Codecs Ceyu Xu Yongji Wu Xinyu Yang Beidi Chen Matthew Lentz Danyang Zhuo Lisa Wu Wills 45 0 0 29 Jun 2024
Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Survey Feng Liang Zhen Zhang Haifeng Lu Chengming Li Victor C. M. Leung Yanyi Guo Xiping Hu 38 3 0 12 Jun 2024
Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging Todd C. Hollon Cheng Jiang Asadur Chowdury Mustafa Nasir-Moin A. Kondepudi ... M. Snuderl S. Camelo-Piragua C. Freudiger Ho Hin Lee D. Orringer 24 86 0 23 Mar 2023
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models Sanjith Athlur Nitika Saran Muthian Sivathanu R. Ramjee Nipun Kwatra GNN 25 62 0 07 Nov 2021
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines Shigang Li Torsten Hoefler GNN AI4CE LRM 77 130 0 14 Jul 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization Ali Ramezani-Kebrya Fartash Faghri Ilya Markov V. Aksenov Dan Alistarh Daniel M. Roy MQ 51 30 0 28 Apr 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 27 65 0 21 Apr 2021
IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko Bokun Wang D. Kovalev Peter Richtárik 65 14 0 16 Feb 2021
An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems A. Abdelmoniem Ahmed Elzanaty Mohamed-Slim Alouini Marco Canini 49 73 0 26 Jan 2021
FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring Qiong Wu Xu Chen Zhi Zhou Junshan Zhang FedML 150 183 0 14 Dec 2020
HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing Deyin Liu Xu Chen Zhi Zhou Qing Ling 24 45 0 22 Mar 2020
Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence Shuiguang Deng Hailiang Zhao Weijia Fang Jianwei Yin Schahram Dustdar Albert Y. Zomaya 63 598 0 02 Sep 2019
Adaptive Federated Learning in Resource Constrained Edge Computing Systems Shiqiang Wang Tiffany Tuor Theodoros Salonidis K. Leung C. Makaya T. He Kevin S. Chan 136 1,663 0 14 Apr 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 119 1,190 0 16 Aug 2016