Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond

13 March 2023

Papers citing "Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond"

19 / 19 papers shown

Title
Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization Shira Vansover-Hager Tomer Koren Roi Livni 20 0 0 13 May 2025
Better Rates for Random Task Orderings in Continual Linear Models Itay Evron Ran Levinstein Matan Schliserman Uri Sherman Tomer Koren Daniel Soudry Nathan Srebro CLL 30 0 0 06 Apr 2025
Randomised Splitting Methods and Stochastic Gradient Descent Luke Shaw Peter A. Whalley 42 1 0 05 Apr 2025
Low-Rank Thinning Annabelle Michael Carrell Albert Gong Abhishek Shetty Raaz Dwivedi Lester W. Mackey 56 0 0 17 Feb 2025
Random Reshuffling for Stochastic Gradient Langevin Dynamics Luke Shaw Peter A. Whalley 75 3 0 28 Jan 2025
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 22 1 0 10 Oct 2024
Demystifying SGD with Doubly Stochastic Gradients Kyurae Kim Joohwan Ko Yian Ma Jacob R. Gardner 48 0 0 03 Jun 2024
On the Last-Iterate Convergence of Shuffling Gradient Methods Zijian Liu Zhengyuan Zhou 26 2 0 12 Mar 2024
Last Iterate Convergence of Incremental Methods and Applications in Continual Learning Xu Cai Jelena Diakonikolas 23 5 0 11 Mar 2024
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 24 2 0 20 Nov 2023
GraB-sampler: Optimal Permutation-based SGD Data Sampler for PyTorch Guanghao Wei GP 19 0 0 28 Sep 2023
Mini-Batch Optimization of Contrastive Loss Jaewoong Cho Kartik K. Sreenivasan Keon Lee Kyunghoo Mun Soheun Yi Jeong-Gwan Lee Anna Lee Jy-yong Sohn Dimitris Papailiopoulos Kangwook Lee SSL 35 7 0 12 Jul 2023
Empirical Risk Minimization with Shuffled SGD: A Primal-Dual Perspective and Improved Bounds Xu Cai Cheuk Yin Lin Jelena Diakonikolas FedML 26 5 0 21 Jun 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 17 4 0 21 Jun 2023
On Convergence of Incremental Gradient for Non-Convex Smooth Functions Anastasia Koloskova N. Doikov Sebastian U. Stich Martin Jaggi 29 2 0 30 May 2023
Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition Chen Fan Christos Thrampoulidis Mark W. Schmidt 20 2 0 02 Apr 2023
On the Training Instability of Shuffling SGD with Batch Normalization David Wu Chulhee Yun S. Sra 24 4 0 24 Feb 2023
Coordinating Distributed Example Orders for Provably Accelerated Training A. Feder Cooper Wentao Guo Khiem Pham Tiancheng Yuan Charlie F. Ruan Yucheng Lu Chris De Sa 29 6 0 02 Feb 2023
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 14 2 0 17 Jul 2022