v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Deep Learning Approach to Diabetic Retinopathy Detection B. Tymchenko Philip Marchenko D. Spodarets 61 164 0 03 Mar 2020
3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks Katia Lupinetti A. Ranieri F. Giannini M. Monti SLR 49 29 0 03 Mar 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 68 3 0 02 Mar 2020
Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction E. Jun A. Mulyadi Jaehun Choi Heung-Il Suk 136 20 0 02 Mar 2020
NeurIPS 2019 Disentanglement Challenge: Improved Disentanglement through Learned Aggregation of Convolutional Feature Maps Maximilian Seitzer Andreas Foltyn Felix P. Kemeth CoGe DRL 8 0 0 27 Feb 2020
Using a thousand optimization tasks to learn hyperparameter search strategies Luke Metz Niru Maheswaranathan Ruoxi Sun C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 117 46 0 27 Feb 2020
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent Bao Wang T. Nguyen Andrea L. Bertozzi Richard G. Baraniuk Stanley J. Osher ODL 77 49 0 24 Feb 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 105 188 0 24 Feb 2020
Learning Dynamic Belief Graphs to Generalize on Text-Based Games Ashutosh Adhikari Xingdi Yuan Marc-Alexandre Côté M. Zelinka Marc-Antoine Rondeau Romain Laroche Pascal Poupart Jian Tang Adam Trischler William L. Hamilton AI4CE 91 81 0 21 Feb 2020
Hierarchical Quantized Autoencoders Will Williams Sam Ringer T. Ash John Hughes D. Macleod Jamie Dougherty BDL DRL 73 66 0 19 Feb 2020
Meta-learning Extractors for Music Source Separation David Samuel Aditya Ganeshan Jason Naradowsky 93 62 0 17 Feb 2020
Electricity Theft Detection with self-attention Paulo Finardi Israel Campiotti Gustavo Plensack R. Souza Rodrigo Nogueira G. Pinheiro R. Lotufo 27 29 0 14 Feb 2020
Liver Segmentation in Abdominal CT Images via Auto-Context Neural Network and Self-Supervised Contour Attention Minyoung Chung Jingyu Lee Jeongjin Lee Y. Shin MedIm 44 29 0 14 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 70 18 0 12 Feb 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 160 1,002 0 12 Feb 2020
Short Term Blood Glucose Prediction based on Continuous Glucose Monitoring Data Ali Mohebbi Alexander R. Johansen Nicklas Hansen Peter Ebert Christensen J. Tarp M. L. Jensen Henrik Bengtsson Morten Mørup 38 24 0 06 Feb 2020
A neural network model that learns differences in diagnosis strategies among radiologists has an improved area under the curve for aneurysm status classification in magnetic resonance angiography image series Y. Tachibana Masataka Nishimori Naoyuki Kitamura Kensuke Umehara Junko Ota T. Obata T. Higashi 16 2 0 03 Feb 2020
SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation Jesse Sun Fatemeh Darbeha M. Zaidi Bo Wang AAML 84 111 0 21 Jan 2020
Data-Driven Permanent Magnet Temperature Estimation in Synchronous Motors with Supervised Machine Learning Wilhelm Kirchgässner Oliver Wallscheid J. Böcker 39 70 0 17 Jan 2020
MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy? Joseph Bethge Christian Bartz Haojin Yang Ying-Cong Chen Christoph Meinel MQ 89 91 0 16 Jan 2020
Invertible Generative Modeling using Linear Rational Splines H. M. Dolatabadi S. Erfani C. Leckie 127 65 0 15 Jan 2020
Hippocampus Segmentation on Epilepsy and Alzheimer's Disease Studies with Multiple Convolutional Neural Networks Diedre Carmo Bruna Silva C. Yasuda Letícia Rittner R. Lotufo 130 45 0 14 Jan 2020
Fine-grained Image Classification and Retrieval by Combining Visual and Locally Pooled Textual Features Andrés Mafla S. Dey Ali Furkan Biten Lluís Gómez Dimosthenis Karatzas 60 26 0 14 Jan 2020
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising Ziyi Yang Chenguang Zhu R. Gmyr Michael Zeng Xuedong Huang Eric Darve 99 61 0 03 Jan 2020
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks Jeffrey O. Zhang Alexander Sax Amir Zamir Leonidas Guibas Jitendra Malik 91 28 0 31 Dec 2019
Leveraging Lead Bias for Zero-shot Abstractive News Summarization Chenguang Zhu Ziyi Yang R. Gmyr Michael Zeng Xuedong Huang 60 19 0 25 Dec 2019
Deep Connectomics Networks: Neural Network Architectures Inspired by Neuronal Networks Nicholas Roberts Dian Ang Yap Vinay Uday Prabhu AI4CE 21 9 0 19 Dec 2019
Attention-Based Face AntiSpoofing of RGB Images, using a Minimal End-2-End Neural Network A. Ghofrani Rahil Mahdian Toroghi Seyed Mojtaba Tabatabaie CVBM 29 2 0 18 Dec 2019
Regularizing Deep Multi-Task Networks using Orthogonal Gradients Mihai Suteu Yike Guo 78 59 0 14 Dec 2019
NASNet: A Neuron Attention Stage-by-Stage Net for Single Image Deraining Xu Qin Zhiling Wang 123 37 0 06 Dec 2019
Learning to Correspond Dynamical Systems N. Kim Zhaoming Xie M. van de Panne 45 8 0 06 Dec 2019
Revisiting Few-Shot Learning for Facial Expression Recognition Anca-Nicoleta Ciubotaru A. Devos Behzad Bozorgtabar Jean-Philippe Thiran M. Gabrani CVBM 63 11 0 05 Dec 2019
Domain-independent Dominance of Adaptive Methods Pedro H. P. Savarese David A. McAllester Sudarshan Babu Michael Maire ODL 82 22 0 04 Dec 2019
EventGAN: Leveraging Large Scale Image Datasets for Event Cameras A. Z. Zhu ZiYun Wang Kaung Khant Kostas Daniilidis GAN 67 46 0 03 Dec 2019
The Group Loss for Deep Metric Learning Ismail Elezi Sebastiano Vascon Alessandro Torcinovich Marcello Pelillo Laura Leal-Taixe 175 51 0 01 Dec 2019
Learning Rate Dropout Huangxing Lin Weihong Zeng Xinghao Ding Yue Huang Yihong Zhuang John Paisley ODL 53 9 0 30 Nov 2019
End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances Marin Toromanoff É. Wirbel Fabien Moutarde OffRL 172 209 0 25 Nov 2019
Technical report: supervised training of convolutional spiking neural networks with PyTorch Romain Zimmer Thomas Pellegrini S. Singh T. Masquelier 71 32 0 22 Nov 2019
KISS: Keeping It Simple for Scene Text Recognition Christian Bartz Joseph Bethge Haojin Yang Christoph Meinel VLM ViT 78 18 0 19 Nov 2019
Convergence Analysis of a Momentum Algorithm with Adaptive Step Size for Non Convex Optimization Anas Barakat Pascal Bianchi 77 12 0 18 Nov 2019
Understanding the Disharmony between Weight Normalization Family and Weight Decay: $ε-$ shifted $L_2$ Regularizer Li Xiang Chen Shuo Xia Yan Yang Jian 59 2 0 14 Nov 2019
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao T. Zhao 137 563 0 08 Nov 2019
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing Haoming Jiang Chen Liang Chong-Jun Wang T. Zhao 61 34 0 07 Nov 2019
An Algorithm for Routing Capsules in All Domains Franz A. Heinsen 24 4 0 02 Nov 2019
Weakly Supervised Multi-Task Learning for Cell Detection and Segmentation Alireza Chamanzar Yao Nie 53 55 0 27 Oct 2019
TreeCaps: Tree-Structured Capsule Networks for Program Source Code Processing Vinoj Jayasundara Nghi D. Q. Bui Lingxiao Jiang David Lo 68 16 0 27 Oct 2019
Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram Ryuichi Yamamoto Eunwoo Song Jae-Min Kim 62 820 0 25 Oct 2019
Filterbank design for end-to-end speech separation Manuel Pariente Samuele Cornell Antoine Deleforge Emmanuel Vincent 106 69 0 23 Oct 2019
Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch Kaiyang Zhou Tao Xiang 147 119 0 22 Oct 2019
Transformers without Tears: Improving the Normalization of Self-Attention Toan Q. Nguyen Julian Salazar 96 231 0 14 Oct 2019