A Continuized View on Nesterov Acceleration for Stochastic Gradient
Descent and Randomized Gossip

v1v2 (latest)

A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip

10 June 2021

Raphael Berthier

Francis R. Bach

Nicolas Flammarion

Pierre Gaillard

Hadrien Hendrikx

Laurent Massoulié

Adrien B. Taylor

ArXiv (abs)PDF HTML

Papers citing "A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip"

12 / 12 papers shown

Title
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 101 0 0 11 Feb 2025
Nesterov acceleration in benignly non-convex landscapes Kanan Gupta Stephan Wojtowytsch 78 2 0 10 Oct 2024
Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods Chanwoong Park Youngchae Cho Insoon Yang 76 1 0 02 Sep 2024
Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems Michal Dereziñski Daniel LeJeune Deanna Needell E. Rebrova 84 4 0 09 May 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 136 1 0 03 Apr 2024
$$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning$ $\textbf{A}^2\textbf{CiD}^2$ : Accelerating Asynchronous Communication in Decentralized Deep Learning Adel Nabli Eugene Belilovsky Edouard Oyallon 74 7 0 14 Jun 2023
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization Jun-Kun Wang Andre Wibisono 76 10 0 15 Feb 2023
DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization Adel Nabli Edouard Oyallon 86 8 0 26 Jul 2022
Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum Kirby Banman Liam Peet-Paré N. Hegde Alona Fyshe Martha White 106 0 0 22 Mar 2022
Acceleration of Gossip Algorithms through the Euler-Poisson-Darboux Equation Raphael Berthier Mufan Li 106 0 0 22 Feb 2022
The Implicit Regularization of Momentum Gradient Descent with Early Stopping Li Wang Yingcong Zhou Zhiguo Fu 105 2 0 14 Jan 2022
Asynchronous speedup in decentralized optimization Mathieu Even Hadrien Hendrikx Laurent Massoulie 50 5 0 07 Jun 2021