Aggregated Momentum: Stability Through Passive Damping

Aggregated Momentum: Stability Through Passive Damping

1 April 2018

Roger C. Grosse

Papers citing "Aggregated Momentum: Stability Through Passive Damping"

16 / 16 papers shown

Title
Diffusion Sampling with Momentum for Mitigating Divergence Artifacts Suttisak Wizadwongsa Worameth Chinchuthakun Pramook Khungurn Amit Raj Supasorn Suwajanakorn DiffM 45 2 0 20 Jul 2023
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers Yineng Chen Z. Li Lefei Zhang Bo Du Hai Zhao 30 4 0 02 Jul 2023
Improving physics-informed neural networks with meta-learned optimization Alexander Bihlo PINN 36 18 0 13 Mar 2023
Multilevel-in-Layer Training for Deep Neural Network Regression Colin Ponce Ruipeng Li Christina Mao P. Vassilevski AI4CE 19 1 0 11 Nov 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
On the Limitations of Stochastic Pre-processing Defenses Yue Gao Ilia Shumailov Kassem Fawaz Nicolas Papernot AAML SILM 36 30 0 19 Jun 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 33 32 0 22 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective Yasong Feng Weiguo Gao 20 0 0 09 Dec 2021
AdaInject: Injection Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks S. Dubey S. H. Shabbeer Basha S. Singh B. B. Chaudhuri ODL 40 9 0 26 Sep 2021
Reverse engineering learned optimizers reveals known and novel mechanisms Niru Maheswaranathan David Sussillo Luke Metz Ruoxi Sun Jascha Narain Sohl-Dickstein 14 21 0 04 Nov 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 31 79 0 17 Sep 2020
Demon: Improved Neural Network Training with Momentum Decay John Chen Cameron R. Wolfe Zhaoqi Li Anastasios Kyrillidis ODL 21 15 0 11 Oct 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 31 718 0 19 Jul 2019
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 81 129 0 16 Oct 2018
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 105 1,152 0 04 Mar 2015