v1v2v3 (latest)

Communication-efficient distributed SGD with Sketching

12 March 2019

Papers citing "Communication-efficient distributed SGD with Sketching"

50 / 57 papers shown

Title
Tight analyses of first-order methods with error feedback Daniel Berg Thomsen Adrien B. Taylor Aymeric Dieuleveut 91 1 0 05 Jun 2025
Lego Sketch: A Scalable Memory-augmented Neural Network for Sketching Data Streams Yuan Feng Yukun Cao Hairu Wang Xike Xie S.Kevin Zhou 20 0 0 26 May 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 114 0 0 11 Nov 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 156 0 0 11 Jun 2024
Private Federated Learning with Autotuned Compression Enayat Ullah Christopher A. Choquette-Choo Peter Kairouz Sewoong Oh FedML 68 8 0 20 Jul 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 101 22 0 01 Feb 2023
Asymptotics of the Sketched Pseudoinverse Daniel LeJeune Pratik V. Patil Hamid Javadi Richard G. Baraniuk Robert Tibshirani 54 10 0 07 Nov 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 48 6 0 25 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Eric Wang 86 3 0 06 Oct 2022
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression Zhize Li Haoyu Zhao Boyue Li Yuejie Chi FedML 81 41 0 20 Jun 2022
Neurotoxin: Durable Backdoors in Federated Learning Zhengming Zhang Ashwinee Panda Linyue Song Yaoqing Yang Michael W. Mahoney Joseph E. Gonzalez Kannan Ramchandran Prateek Mittal FedML 113 138 0 12 Jun 2022
Communication-Efficient Robust Federated Learning with Noisy Labels Junyi Li Jian Pei Heng Huang FedML 83 18 0 11 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 105 13 0 02 Jun 2022
Federated Semi-Supervised Learning with Prototypical Networks Woojun Kim Keondo Park Kihyuk Sohn Raphael Shu Hyung-Sin Kim FedML 72 12 0 27 May 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 144 13 0 26 May 2022
Self-Aware Personalized Federated Learning Huili Chen Jie Ding Eric W. Tramel Shuang Wu Anit Kumar Sahu Salman Avestimehr Tao Zhang FedML 87 27 0 17 Apr 2022
Correlated quantization for distributed mean estimation and optimization A. Suresh Ziteng Sun Jae Hun Ro Felix X. Yu 101 12 0 09 Mar 2022
On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning Hanhan Zhou Tian-Shing Lan Guru Venkataramani Wenbo Ding FedML 79 7 0 27 Jan 2022
Communication-Efficient Distributed SGD with Compressed Sensing Yujie Tang V. Ramanathan Junshan Zhang Na Li FedML 53 8 0 15 Dec 2021
FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning Keyu Yang Lu Chen Zhihao Zeng Yunjun Gao 47 9 0 08 Dec 2021
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters Xiangru Lian Binhang Yuan Xuefeng Zhu Yulong Wang Yongjun He ... Lei Yuan Hai-bo Yu Sen Yang Ce Zhang Ji Liu VLM 94 36 0 10 Nov 2021
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning Ke-shi Ge Yongquan Fu Zhiquan Lai Xiaoge Deng Dongsheng Li 41 2 0 05 Oct 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 114 49 0 19 Aug 2021
ErrorCompensatedX: error compensation for variance reduced algorithms Hanlin Tang Yao Li Ji Liu Ming Yan 83 10 0 04 Aug 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 80 30 0 03 Jul 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 71 4 0 21 May 2021
DRIVE: One-bit Distributed Mean Estimation S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher OOD FedML 154 55 0 18 May 2021
From Distributed Machine Learning to Federated Learning: A Survey Ji Liu Jizhou Huang Yang Zhou Xuhong Li Shilei Ji Haoyi Xiong Dejing Dou FedML OOD 140 262 0 29 Apr 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 79 67 0 21 Apr 2021
1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed Conglong Li A. A. Awan Hanlin Tang Samyam Rajbhandari Yuxiong He 115 33 0 13 Apr 2021
Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience Avishek Ghosh R. Maity A. Mazumdar Kannan Ramchandran FedML 54 5 0 17 Mar 2021
PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization Bingyan Liu Yao Guo Xiangqun Chen FedML 68 85 0 02 Mar 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 95 47 0 28 Feb 2021
Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach Junshan Zhang Na Li M. Dedeoglu FedML 72 41 0 16 Feb 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed Hanlin Tang Shaoduo Gan A. A. Awan Samyam Rajbhandari Conglong Li Xiangru Lian Ji Liu Ce Zhang Yuxiong He AI4CE 97 87 0 04 Feb 2021
CADA: Communication-Adaptive Distributed Adam Tianyi Chen Ziye Guo Yuejiao Sun W. Yin ODL 39 24 0 31 Dec 2020
More Industry-friendly: Federated Learning with High Efficient Design Dingwei Li Qinglong Chang Lixue Pang Yanfang Zhang Xudong Sun Jikun Ding Liang Zhang FedML 33 1 0 16 Dec 2020
Sketchy With a Chance of Adoption: Can Sketch-Based Telemetry Be Ready for Prime Time? Zaoxing Liu Hun Namkung Anup Agarwal Antonis Manousis P. Steenkiste S. Seshan Vyas Sekar 13 3 0 10 Dec 2020
InstaHide's Sample Complexity When Mixing Two Private Images Baihe Huang Zhao Song Runzhou Tao Junze Yin Ruizhe Zhang Danyang Zhuo MIACV 57 9 0 24 Nov 2020
Sketch and Scale: Geo-distributed tSNE and UMAP Viska Wei Nikita Ivkin Vladimir Braverman A. Szalay 26 4 0 11 Nov 2020
Compression Boosts Differentially Private Federated Learning Raouf Kerkouche G. Ács C. Castelluccia P. Genevès FedML 75 30 0 10 Nov 2020
Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism Vipul Gupta Dhruv Choudhary P. T. P. Tang Xiaohan Wei Xing Wang Yuzhen Huang A. Kejariwal Kannan Ramchandran Michael W. Mahoney 75 33 0 18 Oct 2020
DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs Da Zheng Chao Ma Minjie Wang Jinjing Zhou Qidong Su Xiang Song Quan Gan Zheng Zhang George Karypis FedML GNN 71 250 0 11 Oct 2020
HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients Enmao Diao Jie Ding Vahid Tarokh FedML 100 560 0 03 Oct 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 93 23 0 04 Sep 2020
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm Hanlin Tang Shaoduo Gan Samyam Rajbhandari Xiangru Lian Ji Liu Yuxiong He Ce Zhang 75 8 0 26 Aug 2020
LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets Ang Li Jingwei Sun Binghui Wang Lin Duan Sicheng Li Yiran Chen H. Li FedML 82 127 0 07 Aug 2020
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi FedML 74 54 0 04 Aug 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 100 371 0 15 Jul 2020
$WOR and $p$'s: Sketches for $\ell_p$-Sampling Without Replacement$ WOR and $p$ 's: Sketches for $\ell_p$ -Sampling Without Replacement E. Cohen Rasmus Pagh David P. Woodruff 32 11 0 14 Jul 2020