Sparsified SGD with Memory

20 September 2018

Papers citing "Sparsified SGD with Memory"

41 / 141 papers shown

Title
CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning D. Sarkar Sumit Rai Ankur Narang FedML 18 2 0 14 Nov 2020
Sparse Communication for Training Deep Networks Negar Foroutan Martin Jaggi FedML 22 16 0 19 Sep 2020
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM Chaouki Ben Issaid Anis Elgabli Jihong Park M. Bennis Mérouane Debbah FedML 23 13 0 14 Sep 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 45 22 0 04 Sep 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 15 25 0 24 Aug 2020
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi FedML 9 54 0 04 Aug 2020
On the Convergence of SGD with Biased Gradients Ahmad Ajalloeian Sebastian U. Stich 6 83 0 31 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 30 271 0 02 Jul 2020
rTop-k: A Statistical Estimation Approach to Distributed SGD L. P. Barnes Huseyin A. Inan Berivan Isik Ayfer Özgür 19 65 0 21 May 2020
A Federated Learning Framework for Healthcare IoT devices Binhang Yuan Song Ge Wenhui Xing FedML OOD 15 64 0 07 May 2020
Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks Zhishuai Guo Mingrui Liu Zhuoning Yuan Li Shen Wei Liu Tianbao Yang 30 42 0 05 May 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 36 9 0 11 Apr 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 41 491 0 23 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 22 12 0 06 Mar 2020
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Zhize Li D. Kovalev Xun Qian Peter Richtárik FedML AI4CE 21 133 0 26 Feb 2020
Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees Richeng Jin Yufan Huang Xiaofan He H. Dai Tianfu Wu FedML 22 63 0 25 Feb 2020
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability Yikai Yan Chaoyue Niu Yucheng Ding Zhenzhe Zheng Fan Wu Guihai Chen Shaojie Tang Zhihua Wu FedML 36 37 0 18 Feb 2020
COKE: Communication-Censored Decentralized Kernel Learning Ping Xu Yue Wang Xiang Chen Z. Tian 11 20 0 28 Jan 2020
Sparse Weight Activation Training Md Aamir Raihan Tor M. Aamodt 32 72 0 07 Jan 2020
Understanding Top-k Sparsification in Distributed Deep Learning S. Shi X. Chu Ka Chun Cheung Simon See 22 93 0 20 Nov 2019
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees S. Shi Zhenheng Tang Qiang-qiang Wang Kaiyong Zhao X. Chu 11 22 0 20 Nov 2019
Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning XINYAN DAI Xiao Yan Kaiwen Zhou Han Yang K. K. Ng James Cheng Yu Fan FedML 8 47 0 12 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 17 141 0 02 Nov 2019
Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning Anis Elgabli Jihong Park Amrit Singh Bedi Chaouki Ben Issaid M. Bennis Vaneet Aggarwal 19 67 0 23 Oct 2019
Abnormal Client Behavior Detection in Federated Learning Suyi Li Yong Cheng Yang Liu Wei Wang Tianjian Chen AAML 6 134 0 22 Oct 2019
Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Felix Sattler K. Müller Wojciech Samek FedML 40 965 0 04 Oct 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 16 22 0 10 Sep 2019
Unified Optimal Analysis of the (Stochastic) Gradient Method Sebastian U. Stich 21 112 0 09 Jul 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 35 2,278 0 04 Jul 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 17 316 0 31 May 2019
Natural Compression for Distributed Deep Learning Samuel Horváth Chen-Yu Ho L. Horvath Atal Narayan Sahu Marco Canini Peter Richtárik 21 150 0 27 May 2019
Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing Zhi Zhou Xu Chen En Li Liekang Zeng Ke Luo Junshan Zhang 19 1,420 0 24 May 2019
Robust and Communication-Efficient Federated Learning from Non-IID Data Felix Sattler Simon Wiedemann K. Müller Wojciech Samek FedML 18 1,330 0 07 Mar 2019
On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communication Sindri Magnússon H. S. Ghadikolaei Na Li 14 81 0 26 Feb 2019
Quantized Frank-Wolfe: Faster Optimization, Lower Communication, and Projection Free Mingrui Zhang Lin Chen Aryan Mokhtari Hamed Hassani Amin Karbasi 16 8 0 17 Feb 2019
A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks S. Shi Qiang-qiang Wang Kaiyong Zhao Zhenheng Tang Yuxin Wang Xiang Huang Xiaowen Chu 32 134 0 14 Jan 2019
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 19 57 0 25 May 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 41 1,043 0 24 May 2018
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication Felix Sattler Simon Wiedemann K. Müller Wojciech Samek MQ 11 210 0 22 May 2018
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method Simon Lacoste-Julien Mark W. Schmidt Francis R. Bach 124 259 0 10 Dec 2012
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 570 0 08 Dec 2012