Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization

21 June 2018

Tong Zhang

Papers citing "Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization"

29 / 29 papers shown

Title
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 24 0 0 17 Jun 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 27 10 0 30 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 26 7 0 12 May 2023
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning Shiqi He Qifan Yan Feijie Wu Lanjun Wang Mathias Lécuyer Ivan Beschastnikh FedML 36 7 0 03 Dec 2022
Towards Practical Few-shot Federated NLP Dongqi Cai Yaozong Wu Haitao Yuan Shangguang Wang F. Lin Mengwei Xu FedML 23 6 0 01 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
Adaptive Top-K in SGD for Communication-Efficient Distributed Learning Mengzhe Ruan Guangfeng Yan Yuanzhang Xiao Linqi Song Weitao Xu 26 3 0 24 Oct 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 16 10 0 08 May 2022
Distributed Adaptive Learning Under Communication Constraints Marco Carpentiero Vincenzo Matta A. H. Sayed 17 17 0 03 Dec 2021
ErrorCompensatedX: error compensation for variance reduced algorithms Hanlin Tang Yao Li Ji Liu Ming Yan 24 9 0 04 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 178 411 0 14 Jul 2021
Towards Demystifying Serverless Machine Learning Training Jiawei Jiang Shaoduo Gan Yue Liu Fanlin Wang Gustavo Alonso Ana Klimovic Ankit Singla Wentao Wu Ce Zhang 19 121 0 17 May 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 27 66 0 21 Apr 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 23 46 0 28 Feb 2021
Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis Hong Xing Osvaldo Simeone Suzhi Bi 39 92 0 29 Jan 2021
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 27 82 0 07 Dec 2020
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 28 108 0 03 Nov 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 25 161 0 06 Aug 2020
Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems Wentai Wu Ligang He Weiwei Lin Rui Mao 17 78 0 28 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 23 271 0 02 Jul 2020
Communication Efficient Federated Learning with Energy Awareness over Wireless Networks Richeng Jin Xiaofan He H. Dai 24 25 0 15 Apr 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 17 12 0 06 Mar 2020
Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees Richeng Jin Yufan Huang Xiaofan He H. Dai Tianfu Wu FedML 22 63 0 25 Feb 2020
Understanding Top-k Sparsification in Distributed Deep Learning S. Shi X. Chu Ka Chun Cheung Simon See 22 93 0 20 Nov 2019
High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning Yuqing Du Sheng Yang Kaibin Huang 24 99 0 09 Oct 2019
A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks S. Shi Qiang-qiang Wang Kaiyong Zhao Zhenheng Tang Yuxin Wang Xiang Huang Xiaowen Chu 32 134 0 14 Jan 2019
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 16 57 0 25 May 2018