v1v2 (latest)

Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

IEEE Journal on Selected Areas in Information Theory (JSAIT), 2019

6 June 2019

Suhas Diggavi

Papers citing "Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations"

50 / 223 papers shown

Title
Communication Efficient LLM Pre-training with SparseLoCo Amir Sarfi Benjamin Thérien Joel Lidin Eugene Belilovsky 72 1 0 21 Aug 2025
Communication-Efficient Distributed Asynchronous ADMM Sagar Shrestha FedML 72 0 0 17 Aug 2025
FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization Seung-Wook Kim Seongyeol Kim Jiah Kim Seowon Ji Se-Ho Lee FedML MQ 179 0 0 30 Jun 2025
Mobility-Aware Asynchronous Federated Learning with Dynamic Sparsification Jintao Yan Tan Chen Yuxuan Sun Zhaojun Nan Sheng Zhou Z. Niu 129 1 0 08 Jun 2025
MuLoCo: Muon is a practical inner optimizer for DiLoCo Benjamin Thérien Xiaolong Huang Irina Rish Eugene Belilovsky MoE 125 6 0 29 May 2025
FDA-Opt: Communication-Efficient Federated Fine-Tuning of Language Models Michail Theologitis V. Samoladas Antonios Deligiannakis 355 0 0 07 May 2025
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao FedML 250 0 0 28 Apr 2025
Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training Yi Hu Jinhang Zuo Eddie Zhang Bob Iannucci Carlee Joe-Wong 223 1 0 13 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 229 0 0 08 Apr 2025
Secure Aggregation in Federated Learning using Multiparty Homomorphic Encryption Erfan Hosseini Shuangyi Chen Ashish Khisti FedML 93 0 0 01 Mar 2025
NCAirFL: CSI-Free Over-the-Air Federated Learning Based on Non-Coherent Detection Haifeng Wen Nicolò Michelusi Osvaldo Simeone Hong Xing FedML 180 0 0 20 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 364 0 0 11 Nov 2024
$r$ Age- $k$ : Communication-Efficient Federated Learning Using Age FactorAsilomar Conference on Signals, Systems and Computers (ACSSC), 2024 Matin Mortaheb Priyanka Kaswan S. Ulukus FedML 190 1 0 29 Oct 2024
Error Feedback under $(L_0,L_1)$ -Smoothness: Normalization and Momentum Sarit Khirirat Abdurakhmon Sadiev Artem Riabinin Eduard A. Gorbunov Peter Richtárik 237 6 0 22 Oct 2024
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM TrainingNeural Information Processing Systems (NeurIPS), 2024 Jinda Jia Cong Xie Hanlin Lu Daoce Wang Hao Feng ... Baixi Sun Yanghua Peng Zhi-Li Zhang Xin Liu Dingwen Tao MQ 219 9 0 20 Oct 2024
Communication-Efficient Federated Group Distributionally Robust OptimizationNeural Information Processing Systems (NeurIPS), 2024 Zhishuai Guo Tianbao Yang FedML 306 1 0 08 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated LearningInternational Conference on Learning Representations (ICLR), 2024 Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 480 41 0 02 Oct 2024
Mitigating Noise Detriment in Differentially Private Federated Learning with Model Pre-training Huitong Jin Yipeng Zhou Quan Z. Sheng Shiting Wen Laizhong Cui AI4CE 198 0 0 18 Aug 2024
FedFQ: Federated Learning with Fine-Grained Quantization Haowei Li Weiying Xie Hangyu Ye Haonan Qin Shuran Ma Yunsong Li FedML MQ 105 3 0 16 Aug 2024
QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices Juntao Zhao Borui Wan Size Zheng Haibin Lin Yibo Zhu Chuan Wu 160 3 0 02 Jul 2024
Differential error feedback for communication-efficient decentralized learning Roula Nassif Stefan Vlaski Marco Carpentiero Vincenzo Matta Ali H. Sayed 284 2 0 26 Jun 2024
Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offs Haifeng Wen Hong Xing Osvaldo Simeone AI4CE 429 5 0 17 Jun 2024
Minimal Communication-Cost Statistical Learning Romain Chor Abdellatif Zaidi Piotr Krasnowski 124 0 0 12 Jun 2024
Local Methods with Adaptivity via Scaling Saveliy Chezhegov Sergey Skorik Nikolas Khachaturov Danil Shalagin A. Avetisyan Aleksandr Beznosikov Martin Takáč Yaroslav Kholodov Alexander Gasnikov 316 4 0 02 Jun 2024
Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging Michail Theologitis Georgios Frangias Georgios Anestis V. Samoladas Antonios Deligiannakis FedML 330 1 0 31 May 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections Roy Miles Pradyumna Reddy Ismail Elezi Jiankang Deng VLM 214 12 0 28 May 2024
Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and PrivacyIEEE Transactions on Machine Learning in Communications and Networking (IEEE TMLCN), 2024 Feng Wang M. C. Gursoy Senem Velipasalar 179 3 0 15 May 2024
Federated Learning with Only Positive Labels by Exploring Label Correlations Xuming An Dui Wang Li Shen Yong Luo Han Hu Bo Du Yonggang Wen Dacheng Tao FedML 207 2 0 24 Apr 2024
Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning Chong Yu Shuaiqi Shen Maroun Touma Kuan Zhang Hai Zhao FedML 178 12 0 15 Apr 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 255 19 0 09 Apr 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 147 2 0 25 Mar 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 335 6 0 07 Mar 2024
On the Convergence of Federated Learning Algorithms without Data Similarity Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 228 4 0 29 Feb 2024
Communication-Efficient Distributed Learning with Local Immediate Error Compensation Yifei Cheng Li Shen Linli Xu Xun Qian Shiwei Wu Yiming Zhou Tie Zhang Dacheng Tao Enhong Chen 160 1 0 19 Feb 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length CodesIEEE Conference on Computer Communications (INFOCOM), 2024 Xiaoxin Su Yipeng Zhou Laizhong Cui John C. S. Lui Jiangchuan Liu FedML 174 2 0 06 Feb 2024
SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding Chanho Park Namyoon Lee FedML AAML 160 4 0 02 Feb 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image InterpretationIEEE Transactions on Geoscience and Remote Sensing (TGRS), 2023 Weiying Xie Zixuan Wang Haonan Qin Daixun Li Yunsong Li 171 3 0 29 Dec 2023
Cooperative Federated Learning over Ground-to-Satellite Integrated Networks: Joint Local Computation and Data Offloading Dong-Jun Han Seyyedali Hosseinalipour David J. Love Mung Chiang Christopher G. Brinton FedML 173 28 0 23 Dec 2023
Efficient Asynchronous Federated Learning with Sparsification and Quantization Juncheng Jia Ji Liu Chendi Zhou Hao Tian M. Dong Dejing Dou FedML 233 17 0 23 Dec 2023
Sparse Training for Federated Learning with Regularized Error Correction Ran Greidi Kobi Cohen FedML 278 2 0 21 Dec 2023
On the Role of Server Momentum in Federated Learning Jianhui Sun Xidong Wu Heng-Chiao Huang Aidong Zhang FedML 198 18 0 19 Dec 2023
Contractive error feedback for gradient compression Bingcong Li Shuai Zheng Parameswaran Raman Anshumali Shrivastava G. Giannakis 135 0 0 13 Dec 2023
Layered Randomized Quantization for Communication-Efficient and Privacy-Preserving Distributed Learning Guangfeng Yan Tan Li Tian-Shing Lan Kui Wu Linqi Song 195 9 0 12 Dec 2023
Correlation Aware Sparsified Mean Estimation Using Random ProjectionNeural Information Processing Systems (NeurIPS), 2023 Shuli Jiang Pranay Sharma Gauri Joshi 244 2 0 29 Oct 2023
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise Aleksandar Armacki Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 554 10 0 28 Oct 2023
AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term MemoryIEEE Wireless Communications and Networking Conference (WCNC), 2023 Haifeng Wen Hong Xing Osvaldo Simeone 180 2 0 25 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep LearningInternational Conference on Learning Representations (ICLR), 2023 Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 219 3 0 22 Oct 2023
LASER: Linear Compression in Wireless Distributed Optimization Ashok Vardhan Makkuva Marco Bondaschi Thijs Vogels Martin Jaggi Hyeji Kim Michael C. Gastpar 316 6 0 19 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 272 11 0 15 Oct 2023
On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networks Xin Liu Wei Tao Dazhi Zhan Yu Pan Xin Ma Yu Ding Zhisong Pan FedML 198 0 0 09 Oct 2023