v1v2v3v4v5v6 (latest)

Don't Use Large Mini-Batches, Use Local SGD

22 August 2018

Papers citing "Don't Use Large Mini-Batches, Use Local SGD"

30 / 280 papers shown

Title
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication Sebastian U. Stich Sai Praneeth Karimireddy FedML 138 16 0 11 Sep 2019
Addressing Algorithmic Bottlenecks in Elastic Machine Learning with Chicle Michael Kaufmann K. Kourtis Celestine Mendler-Dünner Adrian Schüpbach Thomas Parnell 92 0 0 11 Sep 2019
Tighter Theory for Local SGD on Identical and Heterogeneous DataInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2019 Ahmed Khaled Konstantin Mishchenko Peter Richtárik 351 458 0 10 Sep 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 164 26 0 10 Sep 2019
Hierarchical Federated Learning Across Heterogeneous Cellular NetworksIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019 Mehdi Salehi Heydar Abad Emre Ozfatura Deniz Gunduz Ozgur Ercetin FedML 200 353 0 05 Sep 2019
Federated Learning: Challenges, Methods, and Future DirectionsIEEE Signal Processing Magazine (IEEE SPM), 2019 Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 1.5K 5,326 0 21 Aug 2019
Federated Learning over Wireless Fading ChannelsIEEE Transactions on Wireless Communications (TWC), 2019 M. Amiri Deniz Gunduz 264 571 0 23 Jul 2019
Decentralized Deep Learning with Arbitrary Communication CompressionInternational Conference on Learning Representations (ICLR), 2019 Anastasia Koloskova Tao Lin Sebastian U. Stich Martin Jaggi FedML 303 251 0 22 Jul 2019
Collaborative Machine Learning at the Wireless Edge with Blind TransmittersIEEE Global Conference on Signal and Information Processing (GlobalSIP), 2019 M. Amiri T. Duman Deniz Gunduz 120 57 0 08 Jul 2019
On the Convergence of FedAvg on Non-IID DataInternational Conference on Learning Representations (ICLR), 2019 Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 602 2,695 0 04 Jul 2019
An Accelerated Decentralized Stochastic Proximal Algorithm for Finite SumsNeural Information Processing Systems (NeurIPS), 2019 Aymeric Dieuleveut Francis R. Bach Laurent Massoulie 153 33 0 27 May 2019
Decentralized Bayesian Learning over Graphs Anusha Lalitha Xinghan Wang O. Kilinc Y. Lu T. Javidi F. Koushanfar FedML 178 26 0 24 May 2019
On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex OptimizationInternational Conference on Machine Learning (ICML), 2019 Hao Yu Rong Jin 192 52 0 10 May 2019
On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex OptimizationInternational Conference on Machine Learning (ICML), 2019 Hao Yu Rong Jin Sen Yang FedML 243 406 0 09 May 2019
Communication trade-offs for synchronized distributed SGD with large step size Kumar Kshitij Patel Hadrien Hendrikx FedML 162 27 0 25 Apr 2019
CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks Peng Li Susie Xi Rao Jennifer Blase Yue Zhang Xu Chu Ce Zhang 143 42 0 20 Apr 2019
A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction Fan Zhou Guojing Cong 135 8 0 12 Mar 2019
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air Mohammad Mohammadi Amiri Deniz Gunduz 328 58 0 03 Jan 2019
Federated Optimization in Heterogeneous Networks Tian Li Anit Kumar Sahu Manzil Zaheer Maziar Sanjabi Ameet Talwalkar Virginia Smith FedML 897 6,599 0 14 Dec 2018
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks Kazuki Osawa Yohei Tsuji Yuichiro Ueno Akira Naruse Rio Yokota Satoshi Matsuoka ODL 323 96 0 29 Nov 2018
Measuring the Effects of Data Parallelism on Neural Network TrainingJournal of machine learning research (JMLR), 2018 Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 533 450 0 08 Nov 2018
Elastic CoCoA: Scaling In to Improve Convergence Michael Kaufmann Chia-Wen Cheng K. Kourtis 93 3 0 06 Nov 2018
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD Jianyu Wang Gauri Joshi FedML 172 243 0 19 Oct 2018
Distributed Learning over Unreliable Networks Chen Yu Hanlin Tang Cédric Renggli S. Kassing Ankit Singla Dan Alistarh Ce Zhang Ji Liu OOD 224 66 0 17 Oct 2018
Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation Nikolay Bogoychev Marcin Junczys-Dowmunt Kenneth Heafield Alham Fikri Aji ODL 123 17 0 27 Aug 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 358 355 0 22 Aug 2018
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep LearningAAAI Conference on Artificial Intelligence (AAAI), 2018 Hao Yu Sen Yang Shenghuo Zhu MoMe FedML 434 655 0 17 Jul 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 851 1,173 0 24 May 2018
Communication Compression for Decentralized Training Hanlin Tang Shaoduo Gan Ce Zhang Tong Zhang Ji Liu 319 288 0 17 Mar 2018
Federated Meta-Learning with Fast Convergence and Efficient Communication Fei Chen Mi Luo Zhenhua Dong Zhenguo Li Xiuqiang He FedML 237 434 0 22 Feb 2018