v1v2 (latest)

Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

SIAM Journal of Imaging Sciences (SIIMS), 2020

24 February 2020

Bao Wang

T. Nguyen

Andrea L. Bertozzi

Richard G. Baraniuk

Stanley J. Osher

ODL

ArXiv (abs)PDF HTML

Papers citing "Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent"

22 / 22 papers shown

Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization

Kristi Topollai

A. Choromańska

ODL

417

06 Oct 2025

PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems

378

25 Sep 2025

MomentumSMoE: Integrating Momentum into Sparse Mixture of ExpertsNeural Information Processing Systems (NeurIPS), 2024

R. Teo

Tan M. Nguyen

MoE

245

18 Oct 2024

Resetting the Optimizer in Deep RL: An Empirical StudyNeural Information Processing Systems (NeurIPS), 2023

323

30 Jun 2023

AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural NetworksNeural Networks (Neural Netw.), 2023

Li Shen

Liang Ding

210

01 Mar 2023

Momentum Transformer: Closing the Performance Gap Between Self-attention and Its LinearizationMathematical and Scientific Machine Learning (MSML), 2022

T. Nguyen

Richard G. Baraniuk

Robert M. Kirby

Stanley J. Osher

Bao Wang

372

01 Aug 2022

Last-iterate convergence analysis of stochastic momentum methods for neural networksNeurocomputing (Neurocomputing), 2022

101

30 May 2022

An Adaptive Gradient Method with Energy and MomentumAnnals of Applied Mathematics (AAM), 2022

Hailiang Liu

Xuping Tian

ODL

226

23 Mar 2022

Learning POD of Complex Dynamics Using Heavy-ball Neural ODEsJournal of Scientific Computing (J. Sci. Comput.), 2022

Bao Wang

417

24 Feb 2022

Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization

Tao Sun

Huaming Ling

Zuoqiang Shi

Dongsheng Li

Bao Wang

ODL

211

18 Oct 2021

Improving Transformers with Probabilistic Attention Keys

Richard G. Baraniuk

Stanley J. Osher

261

16 Oct 2021

How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies

Bao Wang

Hedi Xia

T. Nguyen

Stanley Osher

AI4CE

247

13 Oct 2021

AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix CompletionSIAM Journal of Imaging Sciences (SIAM J. Imaging Sci.), 2021

Zhemin Li

Tao Sun

Hongxia Wang

Bao Wang

255

12 Oct 2021

Heavy Ball Neural Ordinary Differential EquationsNeural Information Processing Systems (NeurIPS), 2021

Stanley J. Osher

Bao Wang

227

10 Oct 2021

Accelerated Componentwise Gradient Boosting using Efficient Data Representation and Momentum-based Optimization

Daniel Schalk

J. Herbinger

David Rügamer

249

07 Oct 2021

Accelerated Gradient Descent Learning over Multiple Access Fading ChannelsIEEE Journal on Selected Areas in Communications (JSAC), 2021

Raz Paul

Yuval Friedman

Kobi Cohen

322

26 Jul 2021

Momentum-inspired Low-Rank Coordinate Descent for Diagonally Constrained SDPs

Junhyung Lyle Kim

Jose Antonio Lara Benitez

Taha Toghani

Cameron R. Wolfe

Zhiwei Zhang

Anastasios Kyrillidis

229

16 Jun 2021

Convolutional Neural Network(CNN/ConvNet) in Stock Price Movement Prediction

Kunal Bhardwaj

164

03 Jun 2021

Stability and Generalization of the Decentralized Stochastic Gradient Descent

Tao Sun

Dongsheng Li

Bao Wang

297

02 Feb 2021

Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

Bao Wang

Qiang Ye

ODL

226

03 Dec 2020

SMG: A Shuffling Gradient-Based Method with MomentumInternational Conference on Machine Learning (ICML), 2020

Trang H. Tran

Lam M. Nguyen

Quoc Tran-Dinh

437

24 Nov 2020

Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers

928

195

03 Jul 2020