v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training Haohan Guo Heng Lu Na Hu Chunlei Zhang Shan Yang Lei Xie Jane Polak Scowcroft Dong Yu AAML 68 12 0 03 Dec 2020
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers Huiyu Wang Yukun Zhu Hartwig Adam Alan Yuille Liang-Chieh Chen ViT 152 531 0 01 Dec 2020
Adam $^+$ : A Stochastic Method with Adaptive Variance Reduction Mingrui Liu Wei Zhang Francesco Orabona Tianbao Yang 64 28 0 24 Nov 2020
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective Zeke Xie Zhiqiang Xu Jingzhao Zhang Issei Sato Masashi Sugiyama 87 25 0 23 Nov 2020
Error-Bounded Correction of Noisy Labels Songzhu Zheng Pengxiang Wu A. Goswami Mayank Goswami Dimitris N. Metaxas Chao Chen NoLa 78 119 0 19 Nov 2020
Deep learning in magnetic resonance prostate segmentation: A review and a new perspective David Gillespie Connah Kendrick I. Boon C. Boon T. Rattay Moi Hoon Yap 32 12 0 16 Nov 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 86 2 0 15 Nov 2020
Metastatic Cancer Image Classification Based On Deep Learning Method Guanwen Qiu Xiaobing Yu B. Sun Yunpeng Wang Lipei Zhang MedIm 8 1 0 13 Nov 2020
Low-activity supervised convolutional spiking neural networks applied to speech commands recognition Thomas Pellegrini Romain Zimmer T. Masquelier 93 34 0 13 Nov 2020
Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks David Peer Sebastian Stabinger A. Rodríguez-Sánchez 21 6 0 05 Nov 2020
EAdam Optimizer: How $ε$ Impact Adam Wei Yuan Kai-Xin Gao ODL 25 21 0 04 Nov 2020
Generalization to New Actions in Reinforcement Learning Ayush Jain Andrew Szot Joseph J. Lim AI4CE 94 35 0 03 Nov 2020
Generalized Wasserstein Dice Score, Distributionally Robust Deep Learning, and Ranger for brain tumor segmentation: BraTS 2020 challenge Lucas Fidon Sebastien Ourselin Tom Vercauteren OOD MedIm 74 43 0 03 Nov 2020
Point Transformer Nico Engel Vasileios Belagiannis Klaus C. J. Dietmayer 3DPC 190 2,022 0 02 Nov 2020
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 77 174 0 01 Nov 2020
Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: a BraTS 2020 challenge solution T. Henry Alexandre Carré Marvin Lerousseau Théo Estienne C. Robert Nikos Paragios Eric Deutsch 45 80 0 30 Oct 2020
Parallel waveform synthesis based on generative adversarial networks with voicing-aware conditional discriminators Ryuichi Yamamoto Eunwoo Song Min-Jae Hwang Jae-Min Kim 74 18 0 27 Oct 2020
Generative Tomography Reconstruction Matteo Ronchetti D. Bacciu DiffM 3DV MedIm 19 0 0 26 Oct 2020
Learning Contextualized Knowledge Structures for Commonsense Reasoning Jun Yan Mrigank Raman Aaron Chan Tianyu Zhang Ryan Rossi Handong Zhao Sungchul Kim Nedim Lipka Xiang Ren 306 37 0 24 Oct 2020
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li Chong Chen Jiawei Han VLM 120 54 0 23 Oct 2020
Learning Curves for Analysis of Deep Networks Derek Hoiem Tanmay Gupta Zhizhong Li Michal Shlapentokh-Rothman 82 26 0 21 Oct 2020
Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation Dominik Stallmann Jan Philip Göpfert Julian Schmitz A. Grünberger Barbara Hammer 103 6 0 20 Oct 2020
How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers Yuanhao Xiong Xuanqing Liu Li-Cheng Lan Yang You Si Si Cho-Jui Hsieh OOD 92 1 0 19 Oct 2020
Mixed-Lingual Pre-training for Cross-lingual Summarization Ruochen Xu Chenguang Zhu Yu Shi Michael Zeng Xuedong Huang 57 26 0 18 Oct 2020
XPDNet for MRI Reconstruction: an application to the 2020 fastMRI challenge Zaccharie Ramzi P. Ciuciu Jean-Luc Starck 83 23 0 15 Oct 2020
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Juntang Zhuang Tommy M. Tang Yifan Ding S. Tatikonda Nicha Dvornek X. Papademetris James S. Duncan ODL 232 523 0 15 Oct 2020
Learning to Attack with Fewer Pixels: A Probabilistic Post-hoc Framework for Refining Arbitrary Dense Adversarial Attacks He Zhao Thanh-Tuan Nguyen Trung Le Paul Montague O. Vel Tamas Abraham Dinh Q. Phung AAML 52 2 0 13 Oct 2020
Self-attention aggregation network for video face representation and recognition Ihor Protsenko Taras Lehinevych Dmytro Voitekh Ihor Kroosh Nick Hasty Anthony Johnson CVBM 43 2 0 11 Oct 2020
AEGD: Adaptive Gradient Descent with Energy Hailiang Liu Xuping Tian ODL 55 11 0 10 Oct 2020
A Transformer-based Framework for Multivariate Time Series Representation Learning George Zerveas Srideepika Jayaraman Dhaval Patel A. Bhamidipaty Carsten Eickhoff AI4TS 109 948 0 06 Oct 2020
Data-efficient Online Classification with Siamese Networks and Active Learning Kleanthis Malialis C. Panayiotou Marios M. Polycarpou 45 14 0 04 Oct 2020
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting Sayna Ebrahimi Suzanne Petryk Akash Gokul William Gan Joseph E. Gonzalez Marcus Rohrbach Trevor Darrell CLL 83 47 0 04 Oct 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 93 0 0 02 Oct 2020
Learning an optimal PSF-pair for ultra-dense 3D localization microscopy E. Nehme Boris Ferdman Lucien E. Weiss Tal Naor Daniel Freedman T. Michaeli Y. Shechtman 92 5 0 29 Sep 2020
BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model Junjie Liu Dongchao Wen Deyu Wang Wei Tao Tse-Wei Chen Kinya Osa Masami Kato MQ 44 1 0 29 Sep 2020
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization Xuezhe Ma ODL 85 32 0 28 Sep 2020
Classification and understanding of cloud structures via satellite images with EfficientUNet Tashin Ahmed Noor Hossain Nuri Sabab 124 39 0 27 Sep 2020
Towards General Purpose Geometry-Preserving Single-View Depth Estimation Mikhail Romanov Nikolay Patatkin Anna Vorontsova Sergey Nikolenko Anton Konushin Dmitry Senyushkin MDE 38 3 0 25 Sep 2020
Predicting galaxy spectra from images with hybrid convolutional neural networks John F. Wu J. E. G. Peek 27 9 0 25 Sep 2020
2D-3D Geometric Fusion Network using Multi-Neighbourhood Graph Convolution for RGB-D Indoor Scene Classification Albert Mosella-Montoro Javier Ruiz-Hidalgo 3DPC MDE 143 25 0 23 Sep 2020
Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval Andrés Mafla S. Dey Ali Furkan Biten Lluís Gómez Dimosthenis Karatzas 80 25 0 21 Sep 2020
An Interpretable and Uncertainty Aware Multi-Task Framework for Multi-Aspect Sentiment Analysis Tian Shi Ping Wang Chandan K. Reddy 19 0 0 18 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 197 80 0 17 Sep 2020
UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT Andrei Paraschiv Dumitru-Clementin Cercel M. Dascalu 46 17 0 11 Sep 2020
Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT K. Yan Jinzheng Cai Youjing Zheng Adam P. Harrison D. Jin Youbao Tang Yuxing Tang Lingyun Huang Jing Xiao Le Lu 151 87 0 05 Sep 2020
TreeCaps: Tree-Based Capsule Networks for Source Code Processing Nghi D. Q. Bui Yijun Yu Lingxiao Jiang 3DPC GNN 56 39 0 05 Sep 2020
HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis Jiawei Chen Xu Tan Jian Luan Tao Qin Tie-Yan Liu VLM 102 93 0 03 Sep 2020
Non-Local Musical Statistics as Guides for Audio-to-Score Piano Transcription Kentarou Shibata Eita Nakamura Kazuyoshi Yoshii 40 25 0 28 Aug 2020
Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy Zhuotun Zhu D. Jin K. Yan T. Ho X. Ye Dazhou Guo Chun-Hung Chao Jing Xiao Alan Yuille Le Lu 88 31 0 27 Aug 2020
Discrete Word Embedding for Logical Natural Language Understanding Masataro Asai Zilu Tang 36 3 0 26 Aug 2020