Speeding Up Distributed Machine Learning Using Codes

8 December 2015

Kangwook Lee

Maximilian Lam

Ramtin Pedarsani

Dimitris Papailiopoulos

Kannan Ramchandran

ArXiv PDF HTML

Papers citing "Speeding Up Distributed Machine Learning Using Codes"

50 / 266 papers shown

Title
ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding Hongyi Wang Zachary B. Charles Dimitris Papailiopoulos 16 54 0 28 Jan 2019
Heterogeneity-aware Gradient Coding for Straggler Tolerance Yining Qi Song Guo Bin Tang Ruixuan Li Chengjie Li 24 20 0 27 Jan 2019
Distributed Matrix-Vector Multiplication: A Convolutional Coding Approach A. Das A. Ramamoorthy 24 31 0 25 Jan 2019
Fundamental Limits of Approximate Gradient Coding Sinong Wang Jiashang Liu Ness B. Shroff 67 43 0 23 Jan 2019
Straggler Resilient Serverless Computing Based on Polar Codes Burak Bartan Mert Pilanci 28 13 0 21 Jan 2019
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air Mohammad Mohammadi Amiri Deniz Gunduz 30 53 0 03 Jan 2019
CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplication Tayyebeh Jahani-Nezhad M. Maddah-ali 27 42 0 26 Dec 2018
An Application of Storage-Optimal MatDot Codes for Coded Matrix Multiplication: Fast k-Nearest Neighbors Estimation Utsav Sheth Sanghamitra Dutta Malhar Chaudhari Haewon Jeong Yaoqing Yang J. Kohonen Teemu Roos P. Grover 6 39 0 28 Nov 2018
A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplication Sanghamitra Dutta Ziqian Bai Haewon Jeong Tze Meng Low P. Grover 29 107 0 27 Nov 2018
Distributed Gradient Descent with Coded Partial Gradient Computations Emre Ozfatura S. Ulukus Deniz Gunduz 24 40 0 22 Nov 2018
OverSketch: Approximate Matrix Multiplication for the Cloud Vipul Gupta Ryan Sherman Thomas Courtade Kannan Ramchandran 24 50 0 06 Nov 2018
Erasure coding for distributed matrix multiplication for matrices with bounded entries Li Tang Konstantinos Konstantinidis Tianyi Zhou 26 20 0 06 Nov 2018
Computation Scheduling for Distributed Machine Learning with Straggling Workers Mohammad Mohammadi Amiri Deniz Gunduz FedML 16 3 0 23 Oct 2018
Fault Tolerance in Iterative-Convergent Machine Learning Aurick Qiao Bryon Aragam Bingjing Zhang Eric Xing 26 41 0 17 Oct 2018
A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers A. Severinson Alexandre Graell i Amat E. Rosnes Francisco Lázaro G. Liva 23 10 0 08 Oct 2018
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers Nuwan S. Ferdinand S. Draper 13 19 0 06 Oct 2018
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously Songze Li Mingchao Yu Chien-Sheng Yang A. Avestimehr Sreeram Kannan Pramod Viswanath 42 110 0 27 Sep 2018
Speeding Up Distributed Gradient Descent by Utilizing Non-persistent Stragglers Emre Ozfatura Deniz Gunduz S. Ulukus 21 76 0 07 Aug 2018
On the Fundamental Limits of Coded Data Shuffling for Distributed Machine Learning Adel M. Elmahdy S. Mohajer FedML 19 15 0 11 Jul 2018
Faster Data-access in Large-scale Systems: Network-scale Latency Analysis under General Service-time Distributions Avishek Ghosh Kannan Ramchandran 24 2 0 06 Jul 2018
An Exact Quantized Decentralized Gradient Descent Algorithm Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ramtin Pedarsani 25 123 0 29 Jun 2018
Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation J. Kosaian K. V. Rashmi Shivaram Venkataraman 21 49 0 04 Jun 2018
Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy Qian-long Yu Songze Li Netanel Raviv Seyed Mohammadreza Mousavi Kalan Jinhyun So A. Avestimehr 47 382 0 04 Jun 2018
Gradient Coding via the Stochastic Block Model Zachary B. Charles Dimitris Papailiopoulos 21 16 0 25 May 2018
Polynomially Coded Regression: Optimal Straggler Mitigation via Data Encoding Songze Li Seyed Mohammadreza Mousavi Kalan Qian-long Yu Mahdi Soltanolkotabi A. Avestimehr 34 48 0 24 May 2018
Coded FFT and Its Communication Overhead Haewon Jeong Tze Meng Low P. Grover 23 13 0 24 May 2018
Robust Gradient Descent via Moment Encoding with LDPC Codes R. Maity A. S. Rawat A. Mazumdar 30 41 0 22 May 2018
Compressed Coded Distributed Computing Songze Li M. Maddah-ali A. Avestimehr 24 35 0 05 May 2018
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi 21 141 0 27 Apr 2018
Fundamental Limits of Coded Linear Transform Sinong Wang Jiashang Liu Ness B. Shroff Pengyu Yang 34 38 0 25 Apr 2018
Communication-Aware Scheduling of Serial Tasks for Dispersed Computing Chien-Sheng Yang Ramtin Pedarsani A. Avestimehr 27 40 0 17 Apr 2018
Fundamental Resource Trade-offs for Encoded Distributed Optimization A. Avestimehr Seyed Mohammadreza Mousavi Kalan Mahdi Soltanolkotabi 27 2 0 31 Mar 2018
DRACO: Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Hongyi Wang Zachary B. Charles Dimitris Papailiopoulos 19 243 0 27 Mar 2018
Stochastic Learning under Random Reshuffling with Constant Step-sizes Bicheng Ying Kun Yuan Stefan Vlaski Ali H. Sayed 26 36 0 21 Mar 2018
Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning C. Karakuş Yifan Sun Suhas Diggavi W. Yin 26 52 0 14 Mar 2018
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD Sanghamitra Dutta Gauri Joshi Soumyadip Ghosh Parijat Dube P. Nagpurkar 31 194 0 03 Mar 2018
Communication-Computation Efficient Gradient Coding Min Ye Emmanuel Abbe 26 158 0 09 Feb 2018
Coded Sparse Matrix Multiplication Sinong Wang Jiashang Liu Ness B. Shroff 23 123 0 09 Feb 2018
Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding Qian-long Yu M. Maddah-ali A. Avestimehr 31 39 0 23 Jan 2018
Coded Computing for Distributed Graph Analytics Saurav Prakash Amirhossein Reisizadeh Ramtin Pedarsani A. Avestimehr 18 50 0 17 Jan 2018
Hierarchical Coding for Distributed Computing Hyegyeong Park Kangwook Lee Jy-yong Sohn Changho Suh Jaekyun Moon 26 74 0 15 Jan 2018
Dynamic Heterogeneity-Aware Coded Cooperative Computation at the Edge Yasaman Keshtkarjahromi Yuxuan Xing H. Seferoglu 34 42 0 13 Jan 2018
Near Optimal Coded Data Shuffling for Distributed Learning Mohamed Adel Attia Ravi Tandon FedML 36 30 0 05 Jan 2018
Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers A. Severinson A. Graell i Amat E. Rosnes 35 92 0 21 Dec 2017
Combating Computational Heterogeneity in Large-Scale Distributed Computing via Work Exchange Mohamed Adel Attia Ravi Tandon 27 11 0 22 Nov 2017
Approximate Gradient Coding via Sparse Random Graphs Zachary B. Charles Dimitris Papailiopoulos J. Ellenberg 29 77 0 17 Nov 2017
Straggler Mitigation in Distributed Optimization Through Data Encoding C. Karakuş Yifan Sun Suhas Diggavi W. Yin 34 142 0 14 Nov 2017
Near-Optimal Straggler Mitigation for Distributed Gradient Methods Songze Li Seyed Mohammadreza Mousavi Kalan A. Avestimehr Mahdi Soltanolkotabi 27 97 0 27 Oct 2017
A Sequential Approximation Framework for Coded Distributed Optimization Jingge Zhu Ye Pu Vipul Gupta Claire Tomlin Kannan Ramchandran 25 23 0 24 Oct 2017
Coded Fourier Transform Qian-long Yu M. Maddah-ali A. Avestimehr 40 63 0 17 Oct 2017