v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
CRNNs for Urban Sound Tagging with spatiotemporal context Augustin Arnault Nicolas Riche 58 7 0 24 Aug 2020
Automating the assessment of biofouling in images using expert agreement as a gold standard N. Bloomfield Susan Wei Bartholomew Woodham Peter Wilkinson Andrew Robinson 61 18 0 21 Aug 2020
PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data Diedre Carmo Marcos Piau Israel Campiotti Rodrigo Nogueira R. Lotufo LM&MA 79 52 0 20 Aug 2020
TRU-NET: A Deep Learning Approach to High Resolution Prediction of Rainfall Rilwan A. Adewoyin P. Dueben P. Watson Yulan He Ritabrata Dutta AI4CE 54 70 0 20 Aug 2020
SegCodeNet: Color-Coded Segmentation Masks for Activity Detection from Wearable Cameras Asif Sushmit Partho Ghosh Md. Abrar Istiak Nayeeb Rashid Ahsan Habib Akash Taufiq Hasan 23 3 0 19 Aug 2020
Deep Volumetric Ambient Occlusion Dominik Engel Timo Ropinski 77 22 0 19 Aug 2020
Very Deep Transformers for Neural Machine Translation Xiaodong Liu Kevin Duh Liyuan Liu Jianfeng Gao 87 104 0 18 Aug 2020
Adaptive Hierarchical Hyper-gradient Descent Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran 54 5 0 17 Aug 2020
Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI C. Russo Sidong Liu A. Di Ieva 105 22 0 17 Aug 2020
Null-sampling for Interpretable and Fair Representations T. Kehrenberg Myles Bartlett Oliver Thomas Novi Quadrianto OOD 41 29 0 12 Aug 2020
SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving V. Kumar Marvin Klingner S. Yogamani Stefan Milz Tim Fingscheidt Patrick Mäder MDE 128 81 0 10 Aug 2020
Lane Detection Model Based on Spatio-Temporal Network With Double Convolutional Gated Recurrent Units Jiyong Zhang T. Deng Fei Yan Wenbo Liu 90 2 0 10 Aug 2020
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning Sai Praneeth Karimireddy Martin Jaggi Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 170 218 0 08 Aug 2020
Why to "grow" and "harvest" deep learning models? I. Kulikovskikh Tarzan Legović VLM 24 0 0 08 Aug 2020
Unsupervised Cross-Domain Singing Voice Conversion Adam Polyak Lior Wolf Yossi Adi Yaniv Taigman 58 44 0 06 Aug 2020
HooliGAN: Robust, High Quality Neural Vocoding Ollie McCarthy Zo Ahmed 95 14 0 06 Aug 2020
Learning Visual Representations with Caption Annotations Mert Bulent Sariyildiz J. Perez Diane Larlus VLM SSL 119 162 0 04 Aug 2020
Two-Stage Deep Learning for Accelerated 3D Time-of-Flight MRA without Matched Training Data Hyungjin Chung Eunju Cha Leonard Sunwoo J. C. Ye MedIm 89 26 0 04 Aug 2020
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation Elad Richardson Yuval Alaluf Or Patashnik Yotam Nitzan Yaniv Azar Stav Shapiro Daniel Cohen-Or 210 1,114 0 03 Aug 2020
Single Image Cloud Detection via Multi-Image Fusion Scott Workman M. U. Rafique Hunter Blanton Connor Greenwell Nathan Jacobs 24 5 0 29 Jul 2020
Comparative study of deep learning methods for the automatic segmentation of lung, lesion and lesion type in CT scans of COVID-19 patients Sofie Tilborghs Ine Dirks Lucas Fidon S. Willems Tom Eelbode ... D. Robben D. Smeets F. Maes Jef Vandemeulebroucke P. Suetens OOD 87 28 0 29 Jul 2020
AutoClip: Adaptive Gradient Clipping for Source Separation Networks Prem Seetharaman Gordon Wichern Bryan Pardo Jonathan Le Roux 67 34 0 25 Jul 2020
HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs H. Habi Roy H. Jennings Arnon Netzer MQ 72 66 0 20 Jul 2020
Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining Hao-Hsiang Yang Chao-Han Huck Yang Y. Wang 49 30 0 17 Jul 2020
Negative Pseudo Labeling using Class Proportion for Semantic Segmentation in Pathology Hiroki Tokunaga Brian Kenji Iwana Y. Teramoto Akihiko Yoshizawa Ryoma Bise 135 18 0 16 Jul 2020
UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models V. Kumar S. Yogamani Markus Bach Christian Witt Stefan Milz Patrick Mäder MDE 90 53 0 13 Jul 2020
ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar M. Heinzinger Christian Dallago Ghalia Rehawi Yu Wang ... Tamas B. Fehér Christoph Angerer Martin Steinegger D. Bhowmik B. Rost DRL 89 967 0 13 Jul 2020
BERT Learns (and Teaches) Chemistry Josh Payne Mario Srouji Dian Ang Yap V. Kosaraju 59 10 0 11 Jul 2020
Fast Transformers with Clustered Attention Apoorv Vyas Angelos Katharopoulos Franccois Fleuret 80 155 0 09 Jul 2020
Tracking-by-Trackers with a Distilled and Reinforced Model Matteo Dunnhofer N. Martinel C. Micheloni VOT OffRL 64 4 0 08 Jul 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 217 168 0 03 Jul 2020
Learn Faster and Forget Slower via Fast and Stable Task Adaptation Farshid Varno Lucas May Petry Lisa Di-Jorio Stan Matwin CLL 60 2 0 02 Jul 2020
Weakly-Supervised Segmentation for Disease Localization in Chest X-Ray Images Ostap Viniavskyi Mariia Dobko Oles Dobosevych 64 26 0 01 Jul 2020
Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification Xingyang Ni H. Huttunen CVBM 36 20 0 29 Jun 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 218 1,798 0 29 Jun 2020
Matrix Shuffle-Exchange Networks for Hard 2D Tasks Emīls Ozoliņš Kārlis Freivalds A. Sostaks 27 0 0 29 Jun 2020
Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum Zeke Xie Xinrui Wang Huishuai Zhang Issei Sato Masashi Sugiyama ODL 103 47 0 29 Jun 2020
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision Chen Liang Yue Yu Haoming Jiang Siawpeng Er Ruijia Wang T. Zhao Chao Zhang OffRL 78 240 0 28 Jun 2020
Taming GANs with Lookahead-Minmax Tatjana Chavdarova Matteo Pagliardini Sebastian U. Stich François Fleuret Martin Jaggi GAN 61 27 0 25 Jun 2020
MRI Image Reconstruction via Learning Optimization Using Neural ODEs Eric Z. Chen Terrence Chen Shanhui Sun 125 23 0 24 Jun 2020
Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors Karl Pertsch Oleh Rybkin F. Ebert Chelsea Finn Dinesh Jayaraman Sergey Levine 81 73 0 23 Jun 2020
Scribble2Label: Scribble-Supervised Cell Segmentation via Self-Generating Pseudo-Labels with Consistency Hyeonsoo Lee Won-Ki Jeong 89 115 0 23 Jun 2020
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures Julien Launay Iacopo Poli Franccois Boniface Florent Krzakala 127 64 0 23 Jun 2020
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients Chenfei Zhu Yu Cheng Zhe Gan Furong Huang Jingjing Liu Tom Goldstein ODL 109 2 0 21 Jun 2020
Match $^2$ : A Matching over Matching Model for Similar Question Identification Zizhen Wang Yixing Fan Jiafeng Guo Liu Yang Ruqing Zhang Yanyan Lan Xueqi Cheng Hui Jiang Xiaozhao Wang 122 15 0 21 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 91 83 0 20 Jun 2020
Abstract Diagrammatic Reasoning with Multiplex Graph Networks Duo Wang M. Jamnik Pietro Lio 58 66 0 19 Jun 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 135 76 0 18 Jun 2020
Categorical Normalizing Flows via Continuous Transformations Phillip Lippe E. Gavves BDL 136 44 0 17 Jun 2020
Deep Learning based Segmentation of Fish in Noisy Forward Looking MBES Images J. H. Christensen L. V. Mogensen Ole Ravn 46 17 0 16 Jun 2020