Papers citing 'Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions'

Title
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful Martin Marek Sanae Lotfi Aditya Somasundaram A. Wilson Micah Goldblum LRM 88 3 0 09 Jul 2025
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation Robert M. Gower Guillaume Garrigos Nicolas Loizou Dimitris Oikonomou Konstantin Mishchenko Fabian Schaipp 151 3 0 02 Apr 2025
SGD with memory: fundamental properties and stochastic acceleration Dmitry Yarotsky Maksim Velikanov 197 1 0 05 Oct 2024
The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms Elizabeth Collins-Woodfin Inbar Seroussi Begona García Malaxechebarría Andrew W. Mackenzie Elliot Paquette Courtney Paquette 87 2 0 30 May 2024
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum Anh Dang Reza Babanezhad Sharan Vaswani 107 0 0 12 Jan 2024
High-dimensional limit of one-pass SGD on least squares Elizabeth Collins-Woodfin Elliot Paquette 140 4 0 13 Apr 2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon Atish Agarwala Yann N. Dauphin 116 21 0 17 Feb 2023
Flatter, faster: scaling momentum for optimal speedup of SGD Aditya Cowsik T. Can Paolo Glorioso 166 5 0 28 Oct 2022
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 192 20 0 15 Jun 2022