QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

7 October 2016

Dan Alistarh

Papers citing "QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding"

50 / 72 papers shown

Title
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 65 4 0 01 Aug 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 26 7 0 12 May 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression Avetik G. Karagulyan Peter Richtárik FedML 21 6 0 08 Mar 2023
PolarAir: A Compressed Sensing Scheme for Over-the-Air Federated Learning Michail Gkagkos Krishna R. Narayanan J. Chamberland C. Georghiades 30 0 0 24 Jan 2023
Gossiped and Quantized Online Multi-Kernel Learning Tomàs Ortega Hamid Jafarkhani 21 5 0 24 Jan 2023
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 14 6 0 31 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 25 0 0 14 Oct 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 8 23 0 12 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 43 59 0 02 Aug 2022
Private Federated Submodel Learning with Sparsification Sajani Vithana S. Ulukus FedML 22 10 0 31 May 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 18 9 0 31 May 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 29 31 0 30 May 2022
Tighter Regret Analysis and Optimization of Online Federated Learning Dohyeok Kwon Jonghwan Park Songnam Hong 24 11 0 13 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 16 10 0 08 May 2022
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems Yuanxing Zhang Langshi Chen Siran Yang Man Yuan Hui-juan Yi ... Yong Li Dingyang Zhang Wei Lin Lin Qu Bo Zheng 24 32 0 11 Apr 2022
Optimising Communication Overhead in Federated Learning Using NSGA-II José Á. Morell Z. Dahi Francisco Chicano Gabriel Luque Enrique Alba FedML 22 11 0 01 Apr 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 27 3 0 22 Mar 2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 19 47 0 15 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 13 15 0 05 Feb 2022
Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization Jake B. Perazzone Shiqiang Wang Mingyue Ji Kevin S. Chan FedML 19 71 0 19 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 13 8 0 09 Jan 2022
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats Brian Chmiel Ron Banner Elad Hoffer Hilla Ben Yaacov Daniel Soudry MQ 23 22 0 19 Dec 2021
Optimal Rate Adaption in Federated Learning with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou Jiangchuan Liu FedML 28 38 0 13 Dec 2021
Wyner-Ziv Gradient Compression for Federated Learning Kai Liang Huiru Zhong Haoning Chen Youlong Wu FedML 14 8 0 16 Nov 2021
Solving Multi-Arm Bandit Using a Few Bits of Communication Osama A. Hanna Lin F. Yang Christina Fragouli 16 16 0 11 Nov 2021
Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning Xun Qian Rustem Islamov M. Safaryan Peter Richtárik FedML 19 23 0 02 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 22 14 0 01 Nov 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 17 23 0 18 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 44 44 0 07 Oct 2021
Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Leshang Chen Hongyi Wang S. Davidson Edgar Dobriban FedML 24 1 0 04 Oct 2021
Scalable Average Consensus with Compressed Communications Taha Toghani César A. Uribe 15 7 0 14 Sep 2021
Fast Federated Edge Learning with Overlapped Communication and Computation and Channel-Aware Fair Client Scheduling M. E. Ozfatura Junlin Zhao Deniz Gündüz 14 14 0 14 Sep 2021
Bayesian AirComp with Sign-Alignment Precoding for Wireless Federated Learning Chanhoo Park Seunghoon Lee Namyoon Lee 29 5 0 14 Sep 2021
Fundamental limits of over-the-air optimization: Are analog schemes optimal? Shubham K. Jha Prathamesh Mayekar Himanshu Tyagi 17 7 0 11 Sep 2021
ErrorCompensatedX: error compensation for variance reduced algorithms Hanlin Tang Yao Li Ji Liu Ming Yan 17 9 0 04 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 13 2 0 03 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 37 54 0 02 Aug 2021
Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey Lorenz Kummer AI4CE 32 0 0 28 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 173 411 0 14 Jul 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 16 21 0 02 Jul 2021
FedNL: Making Newton-Type Methods Applicable to Federated Learning M. Safaryan Rustem Islamov Xun Qian Peter Richtárik FedML 14 77 0 05 Jun 2021
Towards Demystifying Serverless Machine Learning Training Jiawei Jiang Shaoduo Gan Yue Liu Fanlin Wang Gustavo Alonso Ana Klimovic Ankit Singla Wentao Wu Ce Zhang 19 121 0 17 May 2021
DP-SIGNSGD: When Efficiency Meets Privacy and Robustness Lingjuan Lyu FedML AAML 12 19 0 11 May 2021
Federated Learning: A Signal Processing Perspective Tomer Gafni Nir Shlezinger Kobi Cohen Yonina C. Eldar H. Vincent Poor FedML 21 128 0 31 Mar 2021
Learned Gradient Compression for Distributed Deep Learning L. Abrahamyan Yiming Chen Giannis Bekoulis Nikos Deligiannis 26 45 0 16 Mar 2021
Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget R. Saha Mert Pilanci Andrea J. Goldsmith 11 5 0 13 Mar 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 14 58 0 25 Feb 2021
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 28 108 0 15 Feb 2021
Sparse-Push: Communication- & Energy-Efficient Decentralized Distributed Learning over Directed & Time-Varying Graphs with non-IID Datasets Sai Aparna Aketi Amandeep Singh J. Rabaey 13 10 0 10 Feb 2021
CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning D. Sarkar Sumit Rai Ankur Narang FedML 16 2 0 14 Nov 2020