MoMo: Momentum Models for Adaptive Learning Rates

MoMo: Momentum Models for Adaptive Learning Rates

12 May 2023

Michael Eickenberg

Robert Mansel Gower

Papers citing "MoMo: Momentum Models for Adaptive Learning Rates"

11 / 11 papers shown

Title
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation Robert M. Gower Guillaume Garrigos Nicolas Loizou Dimitris Oikonomou Konstantin Mishchenko Fabian Schaipp 31 0 0 02 Apr 2025
RecMoDiffuse: Recurrent Flow Diffusion for Human Motion Generation Mirgahney Mohamed Harry Jake Cunningham M. Deisenroth Lourdes Agapito DiffM 33 0 0 11 Jun 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 53 4 0 06 Jun 2024
Enhancing Policy Gradient with the Polyak Step-Size Adaption Yunxiang Li Rui Yuan Chen Fan Mark W. Schmidt Samuel Horváth Robert Mansel Gower Martin Takávc 35 0 0 11 Apr 2024
Batch size invariant Adam Xi Wang Laurence Aitchison 38 2 0 29 Feb 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 24 13 0 08 Feb 2024
SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms Farshed Abdukhakimov Chulu Xiang Dmitry Kamzolov Robert Mansel Gower Martin Takáč 32 2 0 28 Dec 2023
Regularized PolyKervNets: Optimizing Expressiveness and Efficiency for Private Inference in Deep Neural Networks Toluwani Aremu 25 0 0 23 Dec 2023
Stochastic Gradient Descent with Preconditioned Polyak Step-size Farshed Abdukhakimov Chulu Xiang Dmitry Kamzolov Martin Takáč 23 5 0 03 Oct 2023
Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimization Karan N. Chadha Gary Cheng John C. Duchi 49 16 0 07 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020