The Power of Factorial Powers: New Parameter settings for (Stochastic)
Optimization

v1v2v3 (latest)

The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization

1 June 2020

Robert Mansel Gower

ArXiv (abs)PDF HTML

Papers citing "The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization"

6 / 6 papers shown

Title
Why Gradients Rapidly Increase Near the End of Training Aaron Defazio 23 0 0 02 Jun 2025
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation Robert M. Gower Guillaume Garrigos Nicolas Loizou Dimitris Oikonomou Konstantin Mishchenko Fabian Schaipp 83 1 0 02 Apr 2025
The Road Less Scheduled Aaron Defazio Xingyu Yang Yang Harsh Mehta Konstantin Mishchenko Ahmed Khaled Ashok Cutkosky 120 60 0 24 May 2024
MoMo: Momentum Models for Adaptive Learning Rates Fabian Schaipp Ruben Ohana Michael Eickenberg Aaron Defazio Robert Mansel Gower 74 13 0 12 May 2023
Learning-Rate-Free Learning by D-Adaptation Aaron Defazio Konstantin Mishchenko 106 85 0 18 Jan 2023
Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization Aaron Defazio Samy Jelassi ODL 67 69 0 26 Jan 2021