ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1908.03265
  4. Cited By
On the Variance of the Adaptive Learning Rate and Beyond
v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019
Liyuan Liu
Haoming Jiang
Pengcheng He
Weizhu Chen
Xiaodong Liu
Jianfeng Gao
Jiawei Han
    ODL
ArXiv (abs)PDFHTMLGithub (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown
Title
Short Text Pre-training with Extended Token Classification for
  E-commerce Query Understanding
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding
Haoming Jiang
Tianyu Cao
Zheng Li
Cheng-hsin Luo
Xianfeng Tang
Qingyu Yin
Danqing Zhang
R. Goutam
Bing Yin
RALM
69
12
0
08 Oct 2022
SAICL: Student Modelling with Interaction-level Auxiliary Contrastive
  Tasks for Knowledge Tracing and Dropout Prediction
SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction
Jungbae Park
Jinyoung Kim
Soonwoo Kwon
Sang Wan Lee
34
1
0
07 Oct 2022
Neural Matching Fields: Implicit Representation of Matching Fields for
  Visual Correspondence
Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
Sung‐Jin Hong
Jisu Nam
Seokju Cho
Susung Hong
Sangryul Jeon
Dongbo Min
Seung Wook Kim
3DV
94
21
0
06 Oct 2022
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning
  on Heterogeneous Data
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
Yuki Takezawa
Hang Bao
Kenta Niwa
Ryoma Sato
Makoto Yamada
76
20
0
30 Sep 2022
Automatic satellite building construction monitoring
Automatic satellite building construction monitoring
Insaf Ashrapov
D. Malakhov
A. Marchenkov
Anton Lulin
Dani El-Ayyass
20
0
0
29 Sep 2022
Multi-encoder attention-based architectures for sound recognition with
  partial visual assistance
Multi-encoder attention-based architectures for sound recognition with partial visual assistance
Wim Boes
Hugo Van hamme
51
1
0
26 Sep 2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight
  Averaging for Better Generalization
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization
Gábor Melis
MoMe
93
1
0
26 Sep 2022
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering
Chen Zheng
Parisa Kordjamshidi
42
6
0
20 Sep 2022
ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech
ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech
Saeed Ghorbani
Ylva Ferstl
Daniel Holden
N. Troje
M. Carbonneau
121
83
0
15 Sep 2022
Beat Transformer: Demixed Beat and Downbeat Tracking with Dilated
  Self-Attention
Beat Transformer: Demixed Beat and Downbeat Tracking with Dilated Self-Attention
Jingwei Zhao
Gus Xia
Ye Wang
64
19
0
15 Sep 2022
Real-world Video Anomaly Detection by Extracting Salient Features in
  Videos
Real-world Video Anomaly Detection by Extracting Salient Features in Videos
Yudai Watanabe
Makoto Okabe
Y. Harada
Naoji Kashima
AI4TS
38
5
0
14 Sep 2022
Graph Neural Networks for Low-Energy Event Classification &
  Reconstruction in IceCube
Graph Neural Networks for Low-Energy Event Classification & Reconstruction in IceCube
R. Abbasi
M. Ackermann
J. Adams
N. Aggarwal
J. Aguilar
...
S. Yoshida
S. Yu
T. Yuan
Zheng Zhang
P. Zhelnin
70
24
0
07 Sep 2022
Self-supervised multimodal neuroimaging yields predictive
  representations for a spectrum of Alzheimer's phenotypes
Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes
A. Fedorov
Eloy P. T. Geenjaar
Lei Wu
Tristan Sylvain
T. DeRamus
Margaux Luck
Maria B. Misiura
R. Devon Hjelm
Sergey Plis
Vince D. Calhoun
31
3
0
07 Sep 2022
UPAR: Unified Pedestrian Attribute Recognition and Person Retrieval
UPAR: Unified Pedestrian Attribute Recognition and Person Retrieval
Andreas Specker
Mickael Cormier
Jürgen Beyerer
CVBM
86
30
0
06 Sep 2022
Revisiting Outer Optimization in Adversarial Training
Revisiting Outer Optimization in Adversarial Training
Ali Dabouei
Fariborz Taherkhani
Sobhan Soleymani
Nasser M. Nasrabadi
AAML
90
4
0
02 Sep 2022
Incorporating Task-specific Concept Knowledge into Script Learning
Incorporating Task-specific Concept Knowledge into Script Learning
Chenkai Sun
Tie Xu
Chengxiang Zhai
Heng Ji
70
5
0
31 Aug 2022
Pipeline-Invariant Representation Learning for Neuroimaging
Pipeline-Invariant Representation Learning for Neuroimaging
Xinhui Li
A. Fedorov
Mrinal Mathur
A. Abrol
Gregory Kiar
Sergey Plis
Vince D. Calhoun
MedIm
40
1
0
27 Aug 2022
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule
  towards Flatter Local Minima
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima
Hengyu Liu
Qiang Fu
Lun Du
Tiancheng Zhang
Gensitskiy Yu.
Shi Han
Dongmei Zhang
150
3
0
25 Aug 2022
TransNet: Category-Level Transparent Object Pose Estimation
TransNet: Category-Level Transparent Object Pose Estimation
Huijie Zhang
Anthony Opipari
Xiaotong Chen
Jiyue Zhu
Zeren Yu
Odest Chadwicke Jenkins
ViT
53
12
0
22 Aug 2022
Adam Can Converge Without Any Modification On Update Rules
Adam Can Converge Without Any Modification On Update Rules
Yushun Zhang
Congliang Chen
Naichen Shi
Ruoyu Sun
Zhimin Luo
116
70
0
20 Aug 2022
How Should We Evaluate Synthesized Environmental Sounds
How Should We Evaluate Synthesized Environmental Sounds
Yuki Okamoto
Keisuke Imoto
Shinnosuke Takamichi
Takahiro Fukumori
Y. Yamashita
44
0
0
16 Aug 2022
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep
  Models
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Xingyu Xie
Pan Zhou
Huan Li
Zhouchen Lin
Shuicheng Yan
ODL
94
169
0
13 Aug 2022
SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct
  Category-Level Object Pose Estimation
SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation
Ruida Zhang
Yan Di
Fabian Manhardt
F. Tombari
Xiangyang Ji
72
37
0
13 Aug 2022
Exploiting Multiple Sequence Lengths in Fast End to End Training for
  Image Captioning
Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning
J. Hu
Roberto Cavicchioli
Alessandro Capotondi
128
22
0
13 Aug 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot
  Performance of Multilingual Translation
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
Muhammad N. ElNokrashy
Amr Hendy
Mohamed Maher
Mohamed Afify
Hany Awadalla
62
2
0
11 Aug 2022
Flexible Unsupervised Learning for Massive MIMO Subarray Hybrid
  Beamforming
Flexible Unsupervised Learning for Massive MIMO Subarray Hybrid Beamforming
Hamed Hojatian
Jérémy Nadal
J. Frigon
Franccois Leduc-Primeau
37
12
0
10 Aug 2022
Adaptive Learning Rates for Faster Stochastic Gradient Methods
Adaptive Learning Rates for Faster Stochastic Gradient Methods
Samuel Horváth
Konstantin Mishchenko
Peter Richtárik
ODL
63
9
0
10 Aug 2022
Continual Prune-and-Select: Class-incremental learning with specialized
  subnetworks
Continual Prune-and-Select: Class-incremental learning with specialized subnetworks
Aleksandr Dekhovich
David Tax
M. Sluiter
Miguel A. Bessa
CLL
70
21
0
09 Aug 2022
Impact Makes a Sound and Sound Makes an Impact: Sound Guides
  Representations and Explorations
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations
Xufeng Zhao
C. Weber
Muhammad Burhan Hafez
S. Wermter
62
9
0
04 Aug 2022
SGEM: stochastic gradient with energy and momentum
SGEM: stochastic gradient with energy and momentum
Hailiang Liu
Xuping Tian
35
4
0
03 Aug 2022
PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object
  Tracking?
PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?
Aleksandr Kim
Guillem Brasó
Aljosa Osep
Laura Leal-Taixé
3DPC
98
51
0
03 Aug 2022
RBP-Pose: Residual Bounding Box Projection for Category-Level Pose
  Estimation
RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation
Ruida Zhang
Yan Di
Zhiqiang Lou
Fabian Manhardt
F. Tombari
Xiangyang Ji
3DPC
111
48
0
30 Jul 2022
Learning Phone Recognition from Unpaired Audio and Phone Sequences Based
  on Generative Adversarial Network
Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network
Da-Rong Liu
Po-Chun Hsu
Yi-Chen Chen
Sung-Feng Huang
Shun-Po Chuang
Da-Yi Wu
Hung-yi Lee
GAN
67
7
0
29 Jul 2022
PEA: Improving the Performance of ReLU Networks for Free by Using
  Progressive Ensemble Activations
PEA: Improving the Performance of ReLU Networks for Free by Using Progressive Ensemble Activations
Á. Utasi
49
0
0
28 Jul 2022
One-Trimap Video Matting
One-Trimap Video Matting
Hongje Seong
Seoung Wug Oh
Brian L. Price
Euntai Kim
Joon-Young Lee
101
13
0
27 Jul 2022
Moment Centralization based Gradient Descent Optimizers for
  Convolutional Neural Networks
Moment Centralization based Gradient Descent Optimizers for Convolutional Neural Networks
Sumanth Sadu
S. Dubey
S. Sreeja
ODL
67
1
0
19 Jul 2022
CATRE: Iterative Point Clouds Alignment for Category-level Object Pose
  Refinement
CATRE: Iterative Point Clouds Alignment for Category-level Object Pose Refinement
Xingyu Liu
Gu Wang
Yi Li
Xiangyang Ji
3DPC
77
29
0
17 Jul 2022
Current Trends in Deep Learning for Earth Observation: An Open-source
  Benchmark Arena for Image Classification
Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
I. Dimitrovski
Ivan Kitanovski
D. Kocev
Nikola Simidjievski
VLM
101
78
0
14 Jul 2022
Speaker consistency loss and step-wise optimization for semi-supervised
  joint training of TTS and ASR using unpaired text data
Speaker consistency loss and step-wise optimization for semi-supervised joint training of TTS and ASR using unpaired text data
Naoki Makishima
Satoshi Suzuki
Atsushi Ando
Ryo Masumura
246
5
0
11 Jul 2022
Joint Analysis of Acoustic Scenes and Sound Events with Weakly labeled
  Data
Joint Analysis of Acoustic Scenes and Sound Events with Weakly labeled Data
Shunsuke Tsubaki
Keisuke Imoto
Nobutaka Ono
27
2
0
10 Jul 2022
Exploring the sequence length bottleneck in the Transformer for Image
  Captioning
Exploring the sequence length bottleneck in the Transformer for Image Captioning
Jiapeng Hu
Roberto Cavicchioli
Alessandro Capotondi
ViT
68
3
0
07 Jul 2022
A Deep Learning Approach for the solution of Probability Density
  Evolution of Stochastic Systems
A Deep Learning Approach for the solution of Probability Density Evolution of Stochastic Systems
S. Pourtakdoust
Amir H. Khodabakhsh
71
14
0
05 Jul 2022
TMGAN-PLC: Audio Packet Loss Concealment using Temporal Memory
  Generative Adversarial Network
TMGAN-PLC: Audio Packet Loss Concealment using Temporal Memory Generative Adversarial Network
Yuansheng Guan
Guochen Yu
Andong Li
C. Zheng
Jie Wang
112
9
0
04 Jul 2022
TTS-by-TTS 2: Data-selective augmentation for neural speech synthesis
  using ranking support vector machine with variational autoencoder
TTS-by-TTS 2: Data-selective augmentation for neural speech synthesis using ranking support vector machine with variational autoencoder
Eunwoo Song
Ryuichi Yamamoto
Ohsung Kwon
Chan Song
Min-Jae Hwang
Suhyeon Oh
Hyun-Wook Yoon
Jin-Seob Kim
Jae-Min Kim
78
7
0
30 Jun 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary
  X-Y Translations
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi
Shufang Xie
Tao Qin
Hany Awadalla
LRM
91
8
0
30 Jun 2022
Adversarial Multi-Task Learning for Disentangling Timbre and Pitch in
  Singing Voice Synthesis
Adversarial Multi-Task Learning for Disentangling Timbre and Pitch in Singing Voice Synthesis
Tae-Woo Kim
Minguk Kang
Gyeong-Hoon Lee
AAML
164
7
0
23 Jun 2022
Joint Analysis of Acoustic Scenes and Sound Events Based on Multitask
  Learning with Dynamic Weight Adaptation
Joint Analysis of Acoustic Scenes and Sound Events Based on Multitask Learning with Dynamic Weight Adaptation
Kayo Nada
Keisuke Imoto
T. Tsuchiya
42
5
0
21 Jun 2022
A Multi-grained based Attention Network for Semi-supervised Sound Event
  Detection
A Multi-grained based Attention Network for Semi-supervised Sound Event Detection
Ying Hu
Xiujuan Zhu
Yun Li
Hao-Ming Huang
Liang He
56
10
0
21 Jun 2022
TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning
TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning
Jinlin Xiang
Eli Shlizerman
CLL
71
8
0
17 Jun 2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and
  Stronger
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger
Zhiqi Bu
Yu Wang
Sheng Zha
George Karypis
134
72
0
14 Jun 2022
Previous
123...789...161718
Next