Training Deep Neural Networks with Adaptive Momentum Inspired by the
Quadratic Optimization

Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization

18 October 2021

Tao Sun

Dongsheng Li

Bao Wang

Papers citing "Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization"

9 / 9 papers shown

Title
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks Jinping Zou Xiaoge Deng Tao Sun 74 0 0 22 Dec 2024
Score-based Generative Models with Adaptive Momentum Ziqing Wen Xiaoge Deng Ping Luo Tao Sun Dongsheng Li DiffM 37 0 0 22 May 2024
High Probability Bounds for Stochastic Continuous Submodular Maximization Evan Becker Jingdong Gao Ted Zadouri Baharan Mirzasoleiman 20 0 0 20 Mar 2023
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization T. Nguyen Richard G. Baraniuk Robert M. Kirby Stanley J. Osher Bao Wang 21 9 0 01 Aug 2022
Conjugate Gradient Method for Generative Adversarial Networks Hiroki Naganuma Hideaki Iiduka GAN 15 1 0 28 Mar 2022
Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs Justin Baker E. Cherkaev A. Narayan Bao Wang AI4CE 11 4 0 24 Feb 2022
How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies Bao Wang Hedi Xia T. Nguyen Stanley Osher AI4CE 37 10 0 13 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 42 6 0 12 Oct 2021
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 81 129 0 16 Oct 2018