Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling

29 September 2017

Papers citing "Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling"

32 / 32 papers shown

Title
Graph Learning at Scale: Characterizing and Optimizing Pre-Propagation GNNs Zichao Yue Chenhui Deng Zhiru Zhang AI4CE 39 0 0 17 Apr 2025
Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization Quoc Tran-Dinh Trang H. Tran Lam M. Nguyen 42 0 0 29 Oct 2024
Expansive Supervision for Neural Radiance Field Weixiang Zhang Shuzhao Xie Shijia Ge Wei Yao Chen Tang Zhi Wang 43 1 0 12 Sep 2024
I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey Noah Lewis J. L. Bez Suren Byna 62 0 0 16 Apr 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 42 1 0 25 Mar 2024
On the Last-Iterate Convergence of Shuffling Gradient Methods Zijian Liu Zhengyuan Zhou 42 2 0 12 Mar 2024
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 31 1 0 24 Oct 2023
KAKURENBO: Adaptively Hiding Samples in Deep Neural Network Training Truong Thao Nguyen Balazs Gerofi Edgar Josafat Martinez-Noriega Franccois Trahay Mohamed Wahib 32 1 0 16 Oct 2023
On Batching Variable Size Inputs for Training End-to-End Speech Enhancement Systems Philippe Gonzalez T. S. Alstrøm Tobias May 24 9 0 25 Jan 2023
Data-driven and machine-learning based prediction of wave propagation behavior in dam-break flood Changli Li Zheng Han Yan-ge Li Ming Li Wei-dong Wang 16 2 0 19 Sep 2022
Exoshuffle: An Extensible Shuffle Architecture Frank Sifei Luan Stephanie Wang Samyukta Yagati Sean Kim Kenneth Lien Isaac Ong Tony Hong S. Cho Eric Liang Ion Stoica 17 6 0 09 Mar 2022
Distributed Random Reshuffling over Networks Kun-Yen Huang Xiao Li Andre Milzarek Shi Pu Junwen Qiu 41 11 0 31 Dec 2021
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing Tianfeng Liu Yangrui Chen Dan Li Chuan Wu Yibo Zhu Jun He Size Zheng Hongzheng Chen Hongzhi Chen Chuanxiong Guo GNN 34 77 0 16 Dec 2021
Distributed stochastic proximal algorithm with random reshuffling for non-smooth finite-sum optimization Xia Jiang Xianlin Zeng Jian Sun Jie Chen Lihua Xie 20 6 0 06 Nov 2021
The challenge of reproducible ML: an empirical study on the impact of bugs Emilio Rivera-Landos Foutse Khomh Amin Nikanjam 8 5 0 09 Sep 2021
Optimizing Deep Neural Networks through Neuroevolution with Stochastic Gradient Descent Haichao Zhang K. Hao Lei Gao Bing Wei Xue-song Tang 21 12 0 21 Dec 2020
SMG: A Shuffling Gradient-Based Method with Momentum Trang H. Tran Lam M. Nguyen Quoc Tran-Dinh 25 21 0 24 Nov 2020
Evaluation of Federated Learning in Phishing Email Detection Chandra Thapa Jun Tang A. Abuadbba Yansong Gao S. Çamtepe Surya Nepal Mahathir Almashor Yifeng Zheng FedML 25 16 0 27 Jul 2020
Distributed Reinforcement Learning of Targeted Grasping with Active Vision for Mobile Manipulators Yasuhiro Fujita Kota Uenishi Avinash Ummadisingu P. Nagarajan Shimpei Masuda M. Castro 32 18 0 16 Jul 2020
Analyzing and Mitigating Data Stalls in DNN Training Jayashree Mohan Amar Phanishayee Ashish Raniwala Vijay Chidambaram 36 105 0 14 Jul 2020
A Hybrid-Order Distributed SGD Method for Non-Convex Optimization to Balance Communication Overhead, Computational Complexity, and Convergence Rate Naeimeh Omidvar M. Maddah-ali Hamed Mahdavi ODL 25 3 0 27 Mar 2020
Closing the convergence gap of SGD without replacement Shashank Rajput Anant Gupta Dimitris Papailiopoulos 14 61 0 24 Feb 2020
A Unified Convergence Analysis for Shuffling-Type Gradient Methods Lam M. Nguyen Quoc Tran-Dinh Dzung Phan Phuong Ha Nguyen Marten van Dijk 44 78 0 19 Feb 2020
Seizure Prediction Using Bidirectional LSTM Hazrat Ali Feroz Karim Junaid Javed Qureshi Adnan O. M. Abuassba Mohammad Farhad Bulbul OOD 16 8 0 13 Dec 2019
Progressive Compressed Records: Taking a Byte out of Deep Learning Data Michael Kuchnik George Amvrosiadis Virginia Smith 22 9 0 01 Nov 2019
Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM Ashesh Chattopadhyay Pedram Hassanzadeh D. Subramanian AI4CE 24 40 0 20 Jun 2019
Sequenced-Replacement Sampling for Deep Learning C. Ho Dae Hoon Park Wei Yang Yi Chang 24 0 0 19 Oct 2018
LIRS: Enabling efficient machine learning on NVM-based storage via a lightweight implementation of random shuffling Zhi-Lin Ke Hsiang-Yun Cheng Chia-Lin Yang 27 9 0 10 Oct 2018
Decentralized Differentially Private Without-Replacement Stochastic Gradient Descent Richeng Jin Xiaofan He H. Dai FedML 25 2 0 08 Sep 2018
On the Fundamental Limits of Coded Data Shuffling for Distributed Machine Learning Adel M. Elmahdy S. Mohajer FedML 19 15 0 11 Jul 2018
Federated Learning with Non-IID Data Yue Zhao Meng Li Liangzhen Lai Naveen Suda Damon Civin Vikas Chandra FedML 78 2,530 0 02 Jun 2018
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 186 683 0 07 Dec 2010