ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1908.03265
  4. Cited By
On the Variance of the Adaptive Learning Rate and Beyond
v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019
Liyuan Liu
Haoming Jiang
Pengcheng He
Weizhu Chen
Xiaodong Liu
Jianfeng Gao
Jiawei Han
    ODL
ArXiv (abs)PDFHTMLGithub (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown
Title
Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning
Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning
Martin Genzel
Ingo Gühring
Jan Macdonald
M. März
64
27
0
14 Jun 2022
Transformers are Meta-Reinforcement Learners
Transformers are Meta-Reinforcement Learners
Luckeciano C. Melo
OffRL
86
50
0
14 Jun 2022
Simple Cues Lead to a Strong Multi-Object Tracker
Simple Cues Lead to a Strong Multi-Object Tracker
Jenny Seidenschwarz
Guillem Brasó
Victor Castro Serrano
Ismail Elezi
Laura Leal-Taixé
VOT
88
54
0
09 Jun 2022
Signal Propagation in Transformers: Theoretical Perspectives and the
  Role of Rank Collapse
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
Lorenzo Noci
Sotiris Anagnostidis
Luca Biggio
Antonio Orvieto
Sidak Pal Singh
Aurelien Lucchi
108
75
0
07 Jun 2022
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal
  Regression
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression
Wanhua Li
Xiaoke Huang
Zheng Hua Zhu
Yansong Tang
Xiu Li
Jie Zhou
Jiwen Lu
116
34
0
06 Jun 2022
A Control Theoretic Framework for Adaptive Gradient Optimizers in
  Machine Learning
A Control Theoretic Framework for Adaptive Gradient Optimizers in Machine Learning
Kushal Chakrabarti
Nikhil Chopra
ODLAI4CE
123
6
0
04 Jun 2022
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax
  Optimization
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Junchi Yang
Xiang Li
Niao He
ODL
111
22
0
01 Jun 2022
VectorAdam for Rotation Equivariant Geometry Optimization
VectorAdam for Rotation Equivariant Geometry Optimization
Selena Ling
Nicholas Sharp
Alec Jacobson
47
16
0
26 May 2022
Learning What and Where: Disentangling Location and Identity Tracking
  Without Supervision
Learning What and Where: Disentangling Location and Identity Tracking Without Supervision
Manuel Traub
S. Otte
Tobias Menge
Matthias Karlbauer
Jannik Thummel
Martin Volker Butz
115
20
0
26 May 2022
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate
N. H. Phong
A. Santos
B. Ribeiro
75
8
0
20 May 2022
Neural Network Architecture Beyond Width and Depth
Neural Network Architecture Beyond Width and Depth
Zuowei Shen
Haizhao Yang
Shijun Zhang
3DVMDE
114
13
0
19 May 2022
Multimodal Indoor Localisation for Measuring Mobility in Parkinson's
  Disease using Transformers
Multimodal Indoor Localisation for Measuring Mobility in Parkinson's Disease using Transformers
Ferdian Jovan
Ryan McConville
Catherine Morgan
E. Tonkin
Alan Whone
I. Craddock
23
1
0
12 May 2022
Unified Source-Filter GAN with Harmonic-plus-Noise Source Excitation
  Generation
Unified Source-Filter GAN with Harmonic-plus-Noise Source Excitation Generation
Reo Yoneyama
Yi-Chiao Wu
Tomoki Toda
70
14
0
12 May 2022
Individualized Risk Assessment of Preoperative Opioid Use by
  Interpretable Neural Network Regression
Individualized Risk Assessment of Preoperative Opioid Use by Interpretable Neural Network Regression
Yuming Sun
Jian Kang
Chad Brummett
Yi Li
AI4CE
26
2
0
07 May 2022
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for
  Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
Jinhao Jiang
Kun Zhou
Wayne Xin Zhao
Ji-Rong Wen
75
15
0
04 May 2022
StorSeismic: A new paradigm in deep learning for seismic processing
StorSeismic: A new paradigm in deep learning for seismic processing
R. Harsuko
T. Alkhalifah
69
38
0
30 Apr 2022
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification
Marcos V. Conde
Kerem Turgutlu
CLIPVLM
99
99
0
29 Apr 2022
Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using
  Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation
Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation
Ryo Terashima
Ryuichi Yamamoto
Eunwoo Song
Yuma Shirahata
Hyun-Wook Yoon
Jae-Min Kim
Kentaro Tachibana
52
16
0
21 Apr 2022
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs
Liang Chen
Peiyi Wang
Runxin Xu
Tianyu Liu
Zhifang Sui
Baobao Chang
95
14
0
19 Apr 2022
Narcissus: A Practical Clean-Label Backdoor Attack with Limited
  Information
Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information
Yi Zeng
Minzhou Pan
H. Just
Lingjuan Lyu
M. Qiu
R. Jia
AAML
96
181
0
11 Apr 2022
ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less
  Neural Networks
ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks
Xiaoxuan Lou
Guowen Xu
Kangjie Chen
Guanlin Li
Jiwei Li
Tianwei Zhang
OODMQ
60
0
0
07 Apr 2022
How Information on Acoustic Scenes and Sound Events Mutually Benefits
  Event Detection and Scene Classification Tasks
How Information on Acoustic Scenes and Sound Events Mutually Benefits Event Detection and Scene Classification Tasks
Keisuke Imoto
Yuka Komatsu
Shunsuke Tsubaki
Tatsuya Komatsu
50
5
0
05 Apr 2022
The Group Loss++: A deeper look into group loss for deep metric learning
The Group Loss++: A deeper look into group loss for deep metric learning
Ismail Elezi
Jenny Seidenschwarz
Laurin Wagner
Sebastiano Vascon
Alessandro Torcinovich
Marcello Pelillo
Laura Leal-Taixe
71
12
0
04 Apr 2022
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single
  Image
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image
Dejia Xu
Yi Ding
Peihao Wang
Zhiwen Fan
Humphrey Shi
Zhangyang Wang
106
190
0
02 Apr 2022
Learning to Deblur using Light Field Generated and Real Defocus Images
Learning to Deblur using Light Field Generated and Real Defocus Images
Lingyan Ruan
Bin Chen
Jizhou Li
Miuling Lam
71
72
0
01 Apr 2022
Weakly Supervised Patch Label Inference Networks for Efficient Pavement
  Distress Detection and Recognition in the Wild
Weakly Supervised Patch Label Inference Networks for Efficient Pavement Distress Detection and Recognition in the Wild
Sheng Huang
Wenhao Tang
Guixin Huang
Luwen Huangfu
Dan Yang
121
9
0
31 Mar 2022
Exploiting Explainable Metrics for Augmented SGD
Exploiting Explainable Metrics for Augmented SGD
Mahdi S. Hosseini
Mathieu Tuli
Konstantinos N. Plataniotis
AAML
63
3
0
31 Mar 2022
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets
Junyong Lee
Myeonghee Lee
Sunghyun Cho
Seungyong Lee
SupR
68
27
0
28 Mar 2022
A DNN Optimizer that Improves over AdaBelief by Suppression of the
  Adaptive Stepsize Range
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range
Guoqiang Zhang
Kenta Niwa
W. Kleijn
ODL
84
2
0
24 Mar 2022
An Adaptive Gradient Method with Energy and Momentum
An Adaptive Gradient Method with Energy and Momentum
Hailiang Liu
Xuping Tian
ODL
48
9
0
23 Mar 2022
Practical tradeoffs between memory, compute, and performance in learned
  optimizers
Practical tradeoffs between memory, compute, and performance in learned optimizers
Luke Metz
C. Freeman
James Harrison
Niru Maheswaranathan
Jascha Narain Sohl-Dickstein
148
32
0
22 Mar 2022
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
Gu Wang
Fabian Manhardt
Xingyu Liu
Xiangyang Ji
F. Tombari
SSL
81
59
0
19 Mar 2022
ESS: Learning Event-based Semantic Segmentation from Still Images
ESS: Learning Event-based Semantic Segmentation from Still Images
Zhaoning Sun
Nico Messikommer
Daniel Gehrig
Davide Scaramuzza
102
83
0
18 Mar 2022
Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation
Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation
Heecheol Kim
Yoshiyuki Ohmura
Yasuo Kuniyoshi
87
28
0
18 Mar 2022
Integrating Language Guidance into Vision-based Deep Metric Learning
Integrating Language Guidance into Vision-based Deep Metric Learning
Karsten Roth
Oriol Vinyals
Zeynep Akata
VLM
55
29
0
16 Mar 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training
Surrogate Gap Minimization Improves Sharpness-Aware Training
Juntang Zhuang
Boqing Gong
Liangzhe Yuan
Huayu Chen
Hartwig Adam
Nicha Dvornek
S. Tatikonda
James Duncan
Ting Liu
105
157
0
15 Mar 2022
Style Transformer for Image Inversion and Editing
Style Transformer for Image Inversion and Editing
Xueqi Hu
Qiusheng Huang
Zhengyi Shi
Siyuan Li
Changxin Gao
Li Sun
Qingli Li
87
56
0
15 Mar 2022
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided
  Point-wise Voting
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting
Yan Di
Ruida Zhang
Zhiqiang Lou
Fabian Manhardt
Xiangyang Ji
Nassir Navab
F. Tombari
80
122
0
15 Mar 2022
RecursiveMix: Mixed Learning with History
RecursiveMix: Mixed Learning with History
Lingfeng Yang
Xiang Li
Borui Zhao
Renjie Song
Jian Yang
VLM
86
20
0
14 Mar 2022
Optimizer Amalgamation
Optimizer Amalgamation
Tianshu Huang
Tianlong Chen
Sijia Liu
Shiyu Chang
Lisa Amini
Zhangyang Wang
MoMe
69
4
0
12 Mar 2022
Near-optimal Deep Reinforcement Learning Policies from Data for Zone
  Temperature Control
Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control
L. D. Natale
B. Svetozarevic
Philipp Heer
Colin N. Jones
OffRLAI4CE
66
6
0
10 Mar 2022
Rethinking data-driven point spread function modeling with a
  differentiable optical model
Rethinking data-driven point spread function modeling with a differentiable optical model
T. Liaudat
Jean-Luc Starck
M. Kilbinger
P. Frugier
43
12
0
09 Mar 2022
SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning
  Prediction of Synthetic Characters
SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters
Albert Mosella-Montoro
Javier Ruiz-Hidalgo
3DH
184
16
0
09 Mar 2022
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question
  Generation
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation
Jishnu Ray Chowdhury
Debanjan Mahata
Cornelia Caragea
71
2
0
09 Mar 2022
Continuous Self-Localization on Aerial Images Using Visual and Lidar
  Sensors
Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors
F. Fervers
Sebastian Bullinger
C. Bodensteiner
Michael Arens
Rainer Stiefelhagen
42
19
0
07 Mar 2022
A Multi-Scale Time-Frequency Spectrogram Discriminator for GAN-based
  Non-Autoregressive TTS
A Multi-Scale Time-Frequency Spectrogram Discriminator for GAN-based Non-Autoregressive TTS
Haohan Guo
Hui Lu
Xixin Wu
Helen Meng
356
7
0
02 Mar 2022
DeepNet: Scaling Transformers to 1,000 Layers
DeepNet: Scaling Transformers to 1,000 Layers
Hongyu Wang
Shuming Ma
Li Dong
Shaohan Huang
Dongdong Zhang
Furu Wei
MoEAI4CE
136
162
0
01 Mar 2022
Echofilter: A Deep Learning Segmentation Model Improves the Automation,
  Standardization, and Timeliness for Post-Processing Echosounder Data in Tidal
  Energy Streams
Echofilter: A Deep Learning Segmentation Model Improves the Automation, Standardization, and Timeliness for Post-Processing Echosounder Data in Tidal Energy Streams
S. Lowe
L. McGarry
Jessica Douglas
Jason Newport
Sageev Oore
Chris Whidden
Daniel J. Hasselman
39
3
0
19 Feb 2022
Training Robots without Robots: Deep Imitation Learning for
  Master-to-Robot Policy Transfer
Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer
Heecheol Kim
Yoshiyuki Ohmura
Akihiko Nagakubo
Yasuo Kuniyoshi
86
25
0
19 Feb 2022
PEg TRAnsfer Workflow recognition challenge report: Does multi-modal
  data improve recognition?
PEg TRAnsfer Workflow recognition challenge report: Does multi-modal data improve recognition?
Arnaud Huaulmé
Kanako Harada
Quang-Minh Nguyen
Bogyu Park
Seungbum Hong
...
Yuto Ishikawa
Yuriko Harai
Satoshi Kondo
M. Mitsuishi
Pierre Jannin
66
9
0
11 Feb 2022
Previous
123...8910...161718
Next