Random Shuffling Beats SGD after Finite Epochs

26 June 2018

Papers citing "Random Shuffling Beats SGD after Finite Epochs"

30 / 30 papers shown

Title
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 52 0 0 11 Jun 2024
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 45 2 0 20 Nov 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 26 1 0 24 Oct 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 24 4 0 21 Jun 2023
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning Patrik Okanovic R. Waleffe Vasilis Mageirakos Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias Nezihe Merve Gürel Theodoros Rekatsinas DD 53 12 0 28 May 2023
Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond Jaeyoung Cha Jaewook Lee Chulhee Yun 28 23 0 13 Mar 2023
Efficiency Ordering of Stochastic Gradient Descent Jie Hu Vishwaraj Doshi Do Young Eun 31 6 0 15 Sep 2022
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 29 2 0 17 Jul 2022
On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms Lam M. Nguyen Trang H. Tran 32 2 0 13 Jun 2022
GraB: Finding Provably Better Data Permutations than Random Reshuffling Yucheng Lu Wentao Guo Christopher De Sa FedML 26 16 0 22 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 27 10 0 08 May 2022
Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines Alexander Isenko R. Mayer Jeffrey Jedele Hans-Arno Jacobsen 19 23 0 17 Feb 2022
Nesterov Accelerated Shuffling Gradient Method for Convex Optimization Trang H. Tran K. Scheinberg Lam M. Nguyen 40 11 0 07 Feb 2022
MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks R. Waleffe J. Mohoney Theodoros Rekatsinas Shivaram Venkataraman GNN 32 25 0 04 Feb 2022
Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods Amirkeivan Mohtashami Sebastian U. Stich Martin Jaggi 26 13 0 03 Feb 2022
Distributed Random Reshuffling over Networks Kun-Yen Huang Xiao Li Andre Milzarek Shi Pu Junwen Qiu 39 11 0 31 Dec 2021
Convergence of Random Reshuffling Under The Kurdyka-Łojasiewicz Inequality Xiao Li Andre Milzarek Junwen Qiu 24 20 0 10 Oct 2021
Optimal Rates for Random Order Online Optimization Uri Sherman Tomer Koren Yishay Mansour 21 8 0 29 Jun 2021
Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems Itay Safran Ohad Shamir 33 19 0 12 Jun 2021
Can Single-Shuffle SGD be Better than Reshuffling SGD and GD? Chulhee Yun S. Sra Ali Jadbabaie 22 10 0 12 Mar 2021
Permutation-Based SGD: Is Random Optimal? Shashank Rajput Kangwook Lee Dimitris Papailiopoulos 28 14 0 19 Feb 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 35 2 0 04 Jan 2021
SMG: A Shuffling Gradient-Based Method with Momentum Trang H. Tran Lam M. Nguyen Quoc Tran-Dinh 23 21 0 24 Nov 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 36 79 0 17 Sep 2020
Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization Chaobing Song Yong Jiang Yi Ma 53 23 0 18 Jun 2020
SGD with shuffling: optimal rates without component convexity and large epoch requirements Kwangjun Ahn Chulhee Yun S. Sra 16 66 0 12 Jun 2020
Random Reshuffling: Simple Analysis with Vast Improvements Konstantin Mishchenko Ahmed Khaled Peter Richtárik 37 131 0 10 Jun 2020
A Unified Convergence Analysis for Shuffling-Type Gradient Methods Lam M. Nguyen Quoc Tran-Dinh Dzung Phan Phuong Ha Nguyen Marten van Dijk 39 78 0 19 Feb 2020
How Good is SGD with Random Shuffling? Itay Safran Ohad Shamir 14 80 0 31 Jul 2019
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 28 117 0 17 Oct 2018