Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization

21 June 2018

Tong Zhang

Papers citing "Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization"

50 / 120 papers shown

Title
Machine Learning and CPU (Central Processing Unit) Scheduling Co-Optimization over a Network of Computing CentersEngineering applications of artificial intelligence (EAAI), 2025 Mohammadreza Doostmohammadian Zulfiya R. Gabidullina Hamid R. Rabiee 61 0 0 29 Oct 2025
Tight analyses of first-order methods with error feedback Daniel Berg Thomsen Adrien B. Taylor Hadrien Hendrikx 199 1 0 05 Jun 2025
Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism Sameera Ramasinghe Thalaiyasingam Ajanthan Gil Avraham Yan Zuo Alexander Long GNN 305 0 0 02 Jun 2025
$γ$ -FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated LearningIEEE Conference on Computer Communications (IEEE INFOCOM), 2025 Rongwei Lu Yutong Jiang Jinrui Zhang Chunyang Li Yifei Zhu Bin Chen Zhi Wang FedML 256 2 0 18 May 2025
Smoothed Normalization for Efficient Distributed Private Optimization Egor Shulgin Sarit Khirirat Peter Richtárik FedML 328 0 0 20 Feb 2025
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 263 1 0 22 Dec 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 364 0 0 11 Nov 2024
Error Feedback under $(L_0,L_1)$ -Smoothness: Normalization and Momentum Sarit Khirirat Abdurakhmon Sadiev Artem Riabinin Eduard A. Gorbunov Peter Richtárik 237 6 0 22 Oct 2024
FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization Linping Qu Shenghui Song Chi-Ying Tsui MQ FedML 127 5 0 26 Jun 2024
Distributed Training of Large Graph Neural Networks with Variable Communication Rates J. Cerviño Md Asadullah Turja Hesham Mostafa N. Himayat Alejandro Ribeiro GNN AI4CE 258 1 0 25 Jun 2024
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 281 0 0 17 Jun 2024
Minimal Communication-Cost Statistical Learning Romain Chor Abdellatif Zaidi Piotr Krasnowski 124 0 0 12 Jun 2024
Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models Zihan Fang Zheng Lin Zhe Chen Xianhao Chen Yue Gao Yuguang Fang 211 50 0 09 Apr 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 255 19 0 09 Apr 2024
On the Convergence of Federated Learning Algorithms without Data Similarity Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 228 4 0 29 Feb 2024
Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach Bin Wang Jun Fang Hongbin Li Yonina C. Eldar FedML 151 2 0 28 Feb 2024
Communication-Efficient Distributed Learning with Local Immediate Error Compensation Yifei Cheng Li Shen Linli Xu Xun Qian Shiwei Wu Yiming Zhou Tie Zhang Dacheng Tao Enhong Chen 160 1 0 19 Feb 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image InterpretationIEEE Transactions on Geoscience and Remote Sensing (TGRS), 2023 Weiying Xie Zixuan Wang Haonan Qin Daixun Li Yunsong Li 163 3 0 29 Dec 2023
Contractive error feedback for gradient compression Bingcong Li Shuai Zheng Parameswaran Raman Anshumali Shrivastava G. Giannakis 135 0 0 13 Dec 2023
Smoothed Gradient Clipping and Error Feedback for Distributed Optimization under Heavy-Tailed Noise Shuhua Yu D. Jakovetić S. Kar 285 1 0 25 Oct 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 172 1 0 23 Sep 2023
FwdLLM: Efficient FedLLM using Forward Gradient Mengwei Xu Dongqi Cai Yaozong Wu Xiang Li Shangguang Wang FedML 206 34 0 26 Aug 2023
Stochastic Controlled Averaging for Federated Learning with Communication CompressionInternational Conference on Learning Representations (ICLR), 2023 Xinmeng Huang Ping Li Xiaoyun Li 319 243 0 16 Aug 2023
Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN TrainingConference on Machine Learning and Systems (MLSys), 2023 Borui Wan Juntao Zhao Chuan Wu GNN 126 21 0 02 Jun 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 197 14 0 30 May 2023
Sharpened Lazy Incremental Quasi-Newton MethodInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Aakash Lahoti Spandan Senapati K. Rajawat Alec Koppel 279 2 0 26 May 2023
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?Neural Information Processing Systems (NeurIPS), 2023 Yutong He Xinmeng Huang Kun Yuan 218 16 0 25 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 327 9 0 12 May 2023
Compressed Regression over Adaptive NetworksIEEE Transactions on Signal and Information Processing over Networks (TSIPN), 2023 Marco Carpentiero Vincenzo Matta Ali H. Sayed 138 3 0 07 Apr 2023
FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning Yi-Rui Yang Kun Wang Wulu Li FedML 198 6 0 09 Mar 2023
Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity Richeng Jin Xiaofan He C. Zhong Zhaoyang Zhang Tony Q.S. Quek H. Dai FedML 142 2 0 19 Feb 2023
Federated Few-Shot Learning for Mobile NLPACM/IEEE International Conference on Mobile Computing and Networking (MobiCom), 2022 Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 193 20 0 12 Dec 2022
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated LearningConference on Machine Learning and Systems (MLSys), 2022 Shiqi He Qifan Yan Feijie Wu Lanjun Wang Mathias Lécuyer Ivan Beschastnikh FedML 186 11 0 03 Dec 2022
Towards Practical Few-shot Federated NLP Dongqi Cai Yaozong Wu Haitao Yuan Shangguang Wang F. Lin Mengwei Xu FedML 261 5 0 01 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 164 6 0 25 Nov 2022
Adaptive Top-K in SGD for Communication-Efficient Distributed LearningGlobal Communications Conference (GLOBECOM), 2022 Mengzhe Ruan Guangfeng Yan Yuanzhang Xiao Linqi Song Weitao Xu 139 3 0 24 Oct 2022
Personalized Federated Learning with Communication Compression El Houcine Bergou Konstantin Burlachenko Aritra Dutta Peter Richtárik FedML 190 10 0 12 Sep 2022
Quantized Adaptive Subgradient Algorithms and Their Applications Ke Xu Jianqiao Wangni Yifan Zhang Deheng Ye Jiaxiang Wu P. Zhao 110 0 0 11 Aug 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication CompressionNeural Information Processing Systems (NeurIPS), 2022 Xinmeng Huang Yiming Chen W. Yin Kun Yuan 291 31 0 08 Jun 2022
ByteComp: Revisiting Gradient Compression in Distributed Training Zhuang Wang Yanghua Peng Yibo Zhu T. Ng 168 2 0 28 May 2022
FedAdapter: Efficient Federated Learning for Modern NLP Dongqi Cai Yaozong Wu Shangguang Wang F. Lin Mengwei Xu FedML AI4CE 160 34 0 20 May 2022
On Distributed Adaptive Optimization with Gradient CompressionInternational Conference on Learning Representations (ICLR), 2022 Xiaoyun Li Belhal Karimi Ping Li 141 31 0 11 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 242 11 0 08 May 2022
Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent SystemInternational Symposium on Information Theory (ISIT), 2022 Mohamed Ridha Znaidi Gaurav Gupta P. Bogdan FedML 99 2 0 24 Apr 2022
Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning Daniel Becking H. Kirchhoffer G. Tech Paul Haase Karsten Müller H. Schwarz Wojciech Samek FedML 134 4 0 09 Apr 2022
Decentralized Multi-Task Stochastic Optimization With Compressed Communications Navjot Singh Xuanyu Cao Suhas Diggavi Tamer Basar 141 12 0 23 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 140 12 0 07 Dec 2021
Distributed Adaptive Learning Under Communication ConstraintsIEEE Open Journal of Signal Processing (JOSP), 2021 Marco Carpentiero Vincenzo Matta Ali H. Sayed 125 19 0 03 Dec 2021
Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping Junya Chen Sijia Wang Lawrence Carin Chenyang Tao 116 3 0 04 Nov 2021
Permutation Compressors for Provably Faster Distributed Nonconvex Optimization Rafal Szlendak Alexander Tyurin Peter Richtárik 301 39 0 07 Oct 2021