ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1908.03265
  4. Cited By
On the Variance of the Adaptive Learning Rate and Beyond
v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019
Liyuan Liu
Haoming Jiang
Pengcheng He
Weizhu Chen
Xiaodong Liu
Jianfeng Gao
Jiawei Han
    ODL
ArXiv (abs)PDFHTMLGithub (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown
Title
Machine Translation into Low-resource Language Varieties
Machine Translation into Low-resource Language Varieties
Sachin Kumar
Antonios Anastasopoulos
S. Wintner
Yulia Tsvetkov
83
30
0
12 Jun 2021
Modeling Hierarchical Structures with Continuous Recursive Neural
  Networks
Modeling Hierarchical Structures with Continuous Recursive Neural Networks
Jishnu Ray Chowdhury
Cornelia Caragea
76
15
0
10 Jun 2021
Generative Feature-driven Image Replay for Continual Learning
Generative Feature-driven Image Replay for Continual Learning
Kevin Thandiackal
Tiziano Portenier
Andrea Giovannini
M. Gabrani
O. Goksel
CLLVLMDiffM
48
10
0
09 Jun 2021
Vanishing Curvature and the Power of Adaptive Methods in Randomly
  Initialized Deep Networks
Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks
Antonio Orvieto
Jonas Köhler
Dario Pavllo
Thomas Hofmann
Aurelien Lucchi
ODL
60
5
0
07 Jun 2021
Highlighting the Importance of Reducing Research Bias and Carbon
  Emissions in CNNs
Highlighting the Importance of Reducing Research Bias and Carbon Emissions in CNNs
A. Badar
Arnav Varma
Adrian Staniec
Mahmoud Gamal
Omar Magdy
Haris Iqbal
Elahe Arani
Bahram Zonooz
95
7
0
06 Jun 2021
Deep Matching Prior: Test-Time Optimization for Dense Correspondence
Deep Matching Prior: Test-Time Optimization for Dense Correspondence
Sunghwan Hong
Seungryong Kim
98
29
0
06 Jun 2021
NAST: A Non-Autoregressive Generator with Word Alignment for
  Unsupervised Text Style Transfer
NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer
Fei Huang
Zikai Chen
Chen Henry Wu
Qihan Guo
Xiaoyan Zhu
Minlie Huang
69
20
0
04 Jun 2021
Adam in Private: Secure and Fast Training of Deep Neural Networks with
  Adaptive Moment Estimation
Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation
Nuttapong Attrapadung
Koki Hamada
Dai Ikarashi
Ryo Kikuchi
Takahiro Matsuda
Ibuki Mishina
Hiraku Morita
Jacob C. N. Schuldt
62
27
0
04 Jun 2021
Deep Personalized Glucose Level Forecasting Using Attention-based
  Recurrent Neural Networks
Deep Personalized Glucose Level Forecasting Using Attention-based Recurrent Neural Networks
Mohammadreza Armandpour
Brian Kidd
Yu Du
Jianhua Z. Huang
68
15
0
02 Jun 2021
ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX
ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX
Pratik Kayal
Mrinal Anand
Harsh Desai
M. Singh
LMTD
64
12
0
30 May 2021
Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data
  via Differentiable Cross-Approximation
Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation
Mikhail (Misha) Usvyatsov
Anastasia Makarova
R. Ballester-Ripoll
M. Rakhuba
Andreas Krause
Konrad Schindler
86
5
0
29 May 2021
Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient
  adaptive algorithms for neural networks
Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks
Dong-Young Lim
Sotirios Sabanis
97
12
0
28 May 2021
An Online Learning System for Wireless Charging Alignment using
  Surround-view Fisheye Cameras
An Online Learning System for Wireless Charging Alignment using Surround-view Fisheye Cameras
Ashok Dahal
V. Kumar
S. Yogamani
Ciarán Eising
139
12
0
26 May 2021
Learning to Detect Fortified Areas
Learning to Detect Fortified Areas
A. Jørgensen
Jonas Tranberg
41
0
0
26 May 2021
Anomaly Detection By Autoencoder Based On Weighted Frequency Domain Loss
Anomaly Detection By Autoencoder Based On Weighted Frequency Domain Loss
M. Nakanishi
Kazuki Sato
Hideo Terada
42
7
0
21 May 2021
AngularGrad: A New Optimization Technique for Angular Convergence of
  Convolutional Neural Networks
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks
S. K. Roy
Mercedes Eugenia Paoletti
J. Haut
S. Dubey
Purushottam Kar
A. Plaza
B. B. Chaudhuri
ODL
85
18
0
21 May 2021
High-Fidelity and Low-Latency Universal Neural Vocoder based on
  Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform
  Modeling
High-Fidelity and Low-Latency Universal Neural Vocoder based on Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform Modeling
Patrick Lumban Tobing
Tomoki Toda
62
8
0
20 May 2021
Generic Itemset Mining Based on Reinforcement Learning
Generic Itemset Mining Based on Reinforcement Learning
Kazuma Fujioka
Kimiaki Shirahama
30
3
0
17 May 2021
On the Distributional Properties of Adaptive Gradients
On the Distributional Properties of Adaptive Gradients
Z. Zhiyi
Liu Ziyin
48
4
0
15 May 2021
Sampling-Frequency-Independent Audio Source Separation Using Convolution
  Layer Based on Impulse Invariant Method
Sampling-Frequency-Independent Audio Source Separation Using Convolution Layer Based on Impulse Invariant Method
Koichi Saito
Tomohiko Nakamura
Kohei Yatabe
Yuma Koizumi
Hiroshi Saruwatari
BDLVLM
36
7
0
10 May 2021
Body Meshes as Points
Body Meshes as Points
Jianfeng Zhang
Dongdong Yu
Jun Hao Liew
Xuecheng Nie
Jiashi Feng
3DH
91
65
0
06 May 2021
Audio Retrieval with Natural Language Queries
Audio Retrieval with Natural Language Queries
Andreea-Maria Oncescu
A. Sophia Koepke
João F. Henriques
Zeynep Akata
Samuel Albanie
63
79
0
05 May 2021
Non-Autoregressive vs Autoregressive Neural Networks for System
  Identification
Non-Autoregressive vs Autoregressive Neural Networks for System Identification
Daniel Weber
C. Gühmann
54
7
0
05 May 2021
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific
  Literature Parsing Task B: Table Recognition to HTML
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML
Jiaquan Ye
Xianbiao Qi
Yelin He
Yihao Chen
Dengyi Gu
Peng Gao
Rong Xiao
LMTD
90
50
0
05 May 2021
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table
  Image Recognition to Latex
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table Image Recognition to Latex
Yelin He
Xianbiao Qi
Jiaquan Ye
Peng Gao
Yihao Chen
Bingcong Li
Xin Tang
Rong Xiao
LMTD
50
11
0
05 May 2021
Joint Registration and Segmentation via Multi-Task Learning for Adaptive
  Radiotherapy of Prostate Cancer
Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer
Mohamed S. Elmahdy
Laurens Beljaards
Sahar Yousefi
Hessam Sokooti
F. Verbeek
U. A. van der Heide
Marius Staring
75
23
0
05 May 2021
Acoustic Scene Classification Using Multichannel Observation with
  Partially Missing Channels
Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels
Keisuke Imoto
29
8
0
05 May 2021
FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive
  Optimizers by Exploiting Strong Convexity
FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity
Yangfan Zhou
Kaizhu Huang
Cheng Cheng
Xuguang Wang
Amir Hussain
Xin Liu
ODL
77
10
0
28 Apr 2021
Learning from Event Cameras with Sparse Spiking Convolutional Neural
  Networks
Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks
Loic Cordone
Benoit Miramond
Sonia Ferrante
98
37
0
26 Apr 2021
VM-MODNet: Vehicle Motion aware Moving Object Detection for Autonomous
  Driving
VM-MODNet: Vehicle Motion aware Moving Object Detection for Autonomous Driving
Hazem Rashed
Ahmad El-Sallab
S. Yogamani
43
4
0
22 Apr 2021
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
Tianyi Wei
Dongdong Chen
Wenbo Zhou
Jing Liao
Weiming Zhang
Lu Yuan
Gang Hua
Nenghai Yu
107
64
0
15 Apr 2021
Spectrogram Inpainting for Interactive Generation of Instrument Sounds
Spectrogram Inpainting for Interactive Generation of Instrument Sounds
Théis Bazin
Gaëtan Hadjeres
P. Esling
M. Malt
61
11
0
15 Apr 2021
RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters
RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters
Daniel Weber
C. Gühmann
Thomas Seel
69
37
0
15 Apr 2021
TransferNet: An Effective and Transparent Framework for Multi-hop
  Question Answering over Relation Graph
TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph
Jiaxin Shi
S. Cao
Lei Hou
Juan-Zi Li
Hanwang Zhang
GNN
90
112
0
15 Apr 2021
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question
  Answering
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
Michihiro Yasunaga
Hongyu Ren
Antoine Bosselut
Percy Liang
J. Leskovec
RALMLMTDAI4MHLRM
92
599
0
13 Apr 2021
Restoring and Mining the Records of the Joseon Dynasty via Neural
  Language Modeling and Machine Translation
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation
Kyeongpil Kang
Kyohoon Jin
Soyoung Yang
Show-Ling Jang
Jaegul Choo
Yougbin Kim
MU
112
18
0
13 Apr 2021
Semantic Segmentation with Generative Models: Semi-Supervised Learning
  and Strong Out-of-Domain Generalization
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
Daiqing Li
Junlin Yang
Karsten Kreis
Antonio Torralba
Sanja Fidler
GANMedIm
110
188
0
12 Apr 2021
Unified Source-Filter GAN: Unified Source-filter Network Based On
  Factorization of Quasi-Periodic Parallel WaveGAN
Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN
Reo Yoneyama
Yi-Chiao Wu
Tomoki Toda
73
12
0
10 Apr 2021
SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround
  View Fisheye Cameras
SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras
Varun Ravi Kumar
Marvin Klingner
S. Yogamani
Markus Bach
Stefan Milz
Tim Fingscheidt
Patrick Mäder
MDE
101
38
0
09 Apr 2021
Fourier Image Transformer
Fourier Image Transformer
T. Buchholz
Florian Jug
ViT
43
19
0
06 Apr 2021
SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model
SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model
Edresson Casanova
C. Shulby
Eren Golge
Nicolas Müller
F. S. Oliveira
Arnaldo Cândido Júnior
A. S. Soares
S. Aluísio
M. Ponti
63
100
0
02 Apr 2021
Action-Based Conversations Dataset: A Corpus for Building More In-Depth
  Task-Oriented Dialogue Systems
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems
Derek Chen
Howard Chen
Yi Yang
A. Lin
Zhou Yu
90
70
0
01 Apr 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to
  Improve Generalization
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
Zeke Xie
Li-xin Yuan
Zhanxing Zhu
Masashi Sugiyama
123
30
0
31 Mar 2021
Research of Damped Newton Stochastic Gradient Descent Method for Neural
  Network Training
Research of Damped Newton Stochastic Gradient Descent Method for Neural Network Training
Jingcheng Zhou
Wei Wei
Zhiming Zheng
ODL
19
0
0
31 Mar 2021
Tasting the cake: evaluating self-supervised generalization on
  out-of-distribution multimodal MRI data
Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data
A. Fedorov
Eloy P. T. Geenjaar
Lei Wu
T. DeRamus
Vince D. Calhoun
Sergey Plis
OODSSL
21
7
0
29 Mar 2021
A Practical Survey on Faster and Lighter Transformers
A Practical Survey on Faster and Lighter Transformers
Quentin Fournier
G. Caron
Daniel Aloise
137
104
0
26 Mar 2021
BART based semantic correction for Mandarin automatic speech recognition
  system
BART based semantic correction for Mandarin automatic speech recognition system
Yun Zhao
Xuerui Yang
Jinchao Wang
Yongyu Gao
Chao Yan
Yuanfu Zhou
VLM
70
29
0
26 Mar 2021
Deepfake Forensics via An Adversarial Game
Deepfake Forensics via An Adversarial Game
Zhi Wang
Yiwen Guo
W. Zuo
AAML
66
36
0
25 Mar 2021
Neural Network Controller for Autonomous Pile Loading Revised
Neural Network Controller for Autonomous Pile Loading Revised
Wenyan Yang
N. Strokina
N. Serbenyuk
Joni Pajarinen
R. Ghabcheloo
J. Vihonen
M. M. Aref
Joni-Kristian Kämäräinen
33
7
0
23 Mar 2021
Weighted Neural Tangent Kernel: A Generalized and Improved
  Network-Induced Kernel
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel
Lei Tan
Shutong Wu
Xiaolin Huang
28
2
0
22 Mar 2021
Previous
123...111213...161718
Next