Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

v1v2 (latest)

Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

24 February 2020

Bao Wang

Andrea L. Bertozzi

Richard G. Baraniuk

Stanley J. Osher

ArXiv (abs)PDF HTML

Papers citing "Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent"

12 / 12 papers shown

Title
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks Hao Sun Li Shen Qihuang Zhong Liang Ding Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao 98 34 0 01 Mar 2023
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization T. Nguyen Richard G. Baraniuk Robert M. Kirby Stanley J. Osher Bao Wang 127 9 0 01 Aug 2022
Improving Transformers with Probabilistic Attention Keys Tam Nguyen T. Nguyen Dung D. Le Duy Khuong Nguyen Viet-Anh Tran Richard G. Baraniuk Nhat Ho Stanley J. Osher 129 33 0 16 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 88 6 0 12 Oct 2021
Heavy Ball Neural Ordinary Differential Equations Hedi Xia Vai Suliafu H. Ji T. Nguyen Andrea L. Bertozzi Stanley J. Osher Bao Wang 94 61 0 10 Oct 2021
Accelerated Componentwise Gradient Boosting using Efficient Data Representation and Momentum-based Optimization Daniel Schalk B. Bischl David Rügamer 62 3 0 07 Oct 2021
Accelerated Gradient Descent Learning over Multiple Access Fading Channels Raz Paul Yuval Friedman Kobi Cohen 89 30 0 26 Jul 2021
Momentum-inspired Low-Rank Coordinate Descent for Diagonally Constrained SDPs Junhyung Lyle Kim Jose Antonio Lara Benitez Taha Toghani Cameron R. Wolfe Zhiwei Zhang Anastasios Kyrillidis 62 0 0 16 Jun 2021
Convolutional Neural Network(CNN/ConvNet) in Stock Price Movement Prediction Kunal Bhardwaj 58 3 0 03 Jun 2021
Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum Bao Wang Qiang Ye ODL 99 14 0 03 Dec 2020
SMG: A Shuffling Gradient-Based Method with Momentum Trang H. Tran Lam M. Nguyen Quoc Tran-Dinh 67 22 0 24 Nov 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 213 168 0 03 Jul 2020