On the Variance of the Adaptive Learning Rate and Beyond

v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

14 / 864 papers shown

Title
On Empirical Comparisons of Optimizers for Deep Learning Dami Choi Christopher J. Shallue Zachary Nado Jaehoon Lee Chris J. Maddison George E. Dahl 126 259 0 11 Oct 2019
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 106 70 0 09 Oct 2019
MGBPv2: Scaling Up Multi-Grid Back-Projection Networks Pablo Navarrete Michelini Wenbin Chen Hanwen Liu Dan Zhu 65 7 0 27 Sep 2019
Port-Hamiltonian Approach to Neural Network Training Stefano Massaroli Michael Poli Federico Califano Angela Faragasso Jinkyoo Park Atsushi Yamashita Hajime Asama 55 14 0 06 Sep 2019
On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement Morten Kolbæk Zheng-Hua Tan S. H. Jensen Jesper Jensen AAML 117 131 0 03 Sep 2019
Kinematic Single Vehicle Trajectory Prediction Baselines and Applications with the NGSIM Dataset Jean Pierre Mercat N. Zoghby G. Sandou D. Beauvois Guillermo Pita Gil AI4TS 63 17 0 29 Aug 2019
Raw-to-End Name Entity Recognition in Social Media Liyuan Liu Zihan Wang Jingbo Shang Dandong Yin Heng Ji Xiang Ren Shaowen Wang Jiawei Han 17 3 0 14 Aug 2019
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Yang Liu Samuel Albanie Arsha Nagrani Andrew Zisserman 91 391 0 31 Jul 2019
Training Neural Networks for and by Interpolation Leonard Berrada Andrew Zisserman M. P. Kumar 3DH 74 63 0 13 Jun 2019
DeepShift: Towards Multiplication-Less Neural Networks Mostafa Elhoushi Zihao Chen F. Shafiq Ye Tian Joey Yiwei Li MQ 131 102 0 30 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 76 57 0 28 May 2019
Parabolic Approximation Line Search for DNNs Max Mutschler A. Zell ODL 95 20 0 28 Mar 2019
Neutron: An Implementation of the Transformer Translation Model and its Variants Hongfei Xu Qiuhui Liu 76 19 0 18 Mar 2019
Albumentations: fast and flexible image augmentations A. Buslaev Alex Parinov Eugene Khvedchenya V. Iglovikov Alexandr A Kalinin 195 2,003 0 18 Sep 2018