v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Machine Translation into Low-resource Language Varieties Sachin Kumar Antonios Anastasopoulos S. Wintner Yulia Tsvetkov 83 30 0 12 Jun 2021
Modeling Hierarchical Structures with Continuous Recursive Neural Networks Jishnu Ray Chowdhury Cornelia Caragea 76 15 0 10 Jun 2021
Generative Feature-driven Image Replay for Continual Learning Kevin Thandiackal Tiziano Portenier Andrea Giovannini M. Gabrani O. Goksel CLL VLM DiffM 48 10 0 09 Jun 2021
Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks Antonio Orvieto Jonas Köhler Dario Pavllo Thomas Hofmann Aurelien Lucchi ODL 60 5 0 07 Jun 2021
Highlighting the Importance of Reducing Research Bias and Carbon Emissions in CNNs A. Badar Arnav Varma Adrian Staniec Mahmoud Gamal Omar Magdy Haris Iqbal Elahe Arani Bahram Zonooz 95 7 0 06 Jun 2021
Deep Matching Prior: Test-Time Optimization for Dense Correspondence Sunghwan Hong Seungryong Kim 98 29 0 06 Jun 2021
NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer Fei Huang Zikai Chen Chen Henry Wu Qihan Guo Xiaoyan Zhu Minlie Huang 69 20 0 04 Jun 2021
Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation Nuttapong Attrapadung Koki Hamada Dai Ikarashi Ryo Kikuchi Takahiro Matsuda Ibuki Mishina Hiraku Morita Jacob C. N. Schuldt 62 27 0 04 Jun 2021
Deep Personalized Glucose Level Forecasting Using Attention-based Recurrent Neural Networks Mohammadreza Armandpour Brian Kidd Yu Du Jianhua Z. Huang 68 15 0 02 Jun 2021
ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX Pratik Kayal Mrinal Anand Harsh Desai M. Singh LMTD 64 12 0 30 May 2021
Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation Mikhail (Misha) Usvyatsov Anastasia Makarova R. Ballester-Ripoll M. Rakhuba Andreas Krause Konrad Schindler 86 5 0 29 May 2021
Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks Dong-Young Lim Sotirios Sabanis 97 12 0 28 May 2021
An Online Learning System for Wireless Charging Alignment using Surround-view Fisheye Cameras Ashok Dahal V. Kumar S. Yogamani Ciarán Eising 139 12 0 26 May 2021
Learning to Detect Fortified Areas A. Jørgensen Jonas Tranberg 41 0 0 26 May 2021
Anomaly Detection By Autoencoder Based On Weighted Frequency Domain Loss M. Nakanishi Kazuki Sato Hideo Terada 42 7 0 21 May 2021
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks S. K. Roy Mercedes Eugenia Paoletti J. Haut S. Dubey Purushottam Kar A. Plaza B. B. Chaudhuri ODL 85 18 0 21 May 2021
High-Fidelity and Low-Latency Universal Neural Vocoder based on Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform Modeling Patrick Lumban Tobing Tomoki Toda 62 8 0 20 May 2021
Generic Itemset Mining Based on Reinforcement Learning Kazuma Fujioka Kimiaki Shirahama 30 3 0 17 May 2021
On the Distributional Properties of Adaptive Gradients Z. Zhiyi Liu Ziyin 48 4 0 15 May 2021
Sampling-Frequency-Independent Audio Source Separation Using Convolution Layer Based on Impulse Invariant Method Koichi Saito Tomohiko Nakamura Kohei Yatabe Yuma Koizumi Hiroshi Saruwatari BDL VLM 36 7 0 10 May 2021
Body Meshes as Points Jianfeng Zhang Dongdong Yu Jun Hao Liew Xuecheng Nie Jiashi Feng 3DH 91 65 0 06 May 2021
Audio Retrieval with Natural Language Queries Andreea-Maria Oncescu A. Sophia Koepke João F. Henriques Zeynep Akata Samuel Albanie 63 79 0 05 May 2021
Non-Autoregressive vs Autoregressive Neural Networks for System Identification Daniel Weber C. Gühmann 54 7 0 05 May 2021
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML Jiaquan Ye Xianbiao Qi Yelin He Yihao Chen Dengyi Gu Peng Gao Rong Xiao LMTD 90 50 0 05 May 2021
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table Image Recognition to Latex Yelin He Xianbiao Qi Jiaquan Ye Peng Gao Yihao Chen Bingcong Li Xin Tang Rong Xiao LMTD 50 11 0 05 May 2021
Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer Mohamed S. Elmahdy Laurens Beljaards Sahar Yousefi Hessam Sokooti F. Verbeek U. A. van der Heide Marius Staring 75 23 0 05 May 2021
Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels Keisuke Imoto 29 8 0 05 May 2021
FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity Yangfan Zhou Kaizhu Huang Cheng Cheng Xuguang Wang Amir Hussain Xin Liu ODL 77 10 0 28 Apr 2021
Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks Loic Cordone Benoit Miramond Sonia Ferrante 98 37 0 26 Apr 2021
VM-MODNet: Vehicle Motion aware Moving Object Detection for Autonomous Driving Hazem Rashed Ahmad El-Sallab S. Yogamani 43 4 0 22 Apr 2021
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion Tianyi Wei Dongdong Chen Wenbo Zhou Jing Liao Weiming Zhang Lu Yuan Gang Hua Nenghai Yu 107 64 0 15 Apr 2021
Spectrogram Inpainting for Interactive Generation of Instrument Sounds Théis Bazin Gaëtan Hadjeres P. Esling M. Malt 61 11 0 15 Apr 2021
RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters Daniel Weber C. Gühmann Thomas Seel 69 37 0 15 Apr 2021
TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph Jiaxin Shi S. Cao Lei Hou Juan-Zi Li Hanwang Zhang GNN 90 112 0 15 Apr 2021
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering Michihiro Yasunaga Hongyu Ren Antoine Bosselut Percy Liang J. Leskovec RALM LMTD AI4MH LRM 92 599 0 13 Apr 2021
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation Kyeongpil Kang Kyohoon Jin Soyoung Yang Show-Ling Jang Jaegul Choo Yougbin Kim MU 112 18 0 13 Apr 2021
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization Daiqing Li Junlin Yang Karsten Kreis Antonio Torralba Sanja Fidler GAN MedIm 110 188 0 12 Apr 2021
Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN Reo Yoneyama Yi-Chiao Wu Tomoki Toda 73 12 0 10 Apr 2021
SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras Varun Ravi Kumar Marvin Klingner S. Yogamani Markus Bach Stefan Milz Tim Fingscheidt Patrick Mäder MDE 101 38 0 09 Apr 2021
Fourier Image Transformer T. Buchholz Florian Jug ViT 43 19 0 06 Apr 2021
SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model Edresson Casanova C. Shulby Eren Golge Nicolas Müller F. S. Oliveira Arnaldo Cândido Júnior A. S. Soares S. Aluísio M. Ponti 63 100 0 02 Apr 2021
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems Derek Chen Howard Chen Yi Yang A. Lin Zhou Yu 90 70 0 01 Apr 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 123 30 0 31 Mar 2021
Research of Damped Newton Stochastic Gradient Descent Method for Neural Network Training Jingcheng Zhou Wei Wei Zhiming Zheng ODL 19 0 0 31 Mar 2021
Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data A. Fedorov Eloy P. T. Geenjaar Lei Wu T. DeRamus Vince D. Calhoun Sergey Plis OOD SSL 21 7 0 29 Mar 2021
A Practical Survey on Faster and Lighter Transformers Quentin Fournier G. Caron Daniel Aloise 137 104 0 26 Mar 2021
BART based semantic correction for Mandarin automatic speech recognition system Yun Zhao Xuerui Yang Jinchao Wang Yongyu Gao Chao Yan Yuanfu Zhou VLM 70 29 0 26 Mar 2021
Deepfake Forensics via An Adversarial Game Zhi Wang Yiwen Guo W. Zuo AAML 66 36 0 25 Mar 2021
Neural Network Controller for Autonomous Pile Loading Revised Wenyan Yang N. Strokina N. Serbenyuk Joni Pajarinen R. Ghabcheloo J. Vihonen M. M. Aref Joni-Kristian Kämäräinen 33 7 0 23 Mar 2021
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel Lei Tan Shutong Wu Xiaolin Huang 28 2 0 22 Mar 2021