ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1908.03265
  4. Cited By
On the Variance of the Adaptive Learning Rate and Beyond
v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019
Liyuan Liu
Haoming Jiang
Pengcheng He
Weizhu Chen
Xiaodong Liu
Jianfeng Gao
Jiawei Han
    ODL
ArXiv (abs)PDFHTMLGithub (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown
Title
Out-of-Domain Generalization in Dynamical Systems Reconstruction
Out-of-Domain Generalization in Dynamical Systems Reconstruction
Niclas Alexander Göring
Florian Hess
Manuel Brenner
Zahra Monfared
Daniel Durstewitz
AI4CE
105
16
0
28 Feb 2024
Hierarchical Multi-Relational Graph Representation Learning for
  Large-Scale Prediction of Drug-Drug Interactions
Hierarchical Multi-Relational Graph Representation Learning for Large-Scale Prediction of Drug-Drug Interactions
Mengying Jiang
Guizhong Liu
Yuanchao Su
Weiqiang Jin
Biao Zhao
100
3
0
28 Feb 2024
NIIRF: Neural IIR Filter Field for HRTF Upsampling and Personalization
NIIRF: Neural IIR Filter Field for HRTF Upsampling and Personalization
Yoshiki Masuyama
Gordon Wichern
François Germain
Zexu Pan
Sameer Khurana
Chiori Hori
Jonathan Le Roux
69
3
0
27 Feb 2024
Why Transformers Need Adam: A Hessian Perspective
Why Transformers Need Adam: A Hessian Perspective
Yushun Zhang
Congliang Chen
Tian Ding
Ziniu Li
Ruoyu Sun
Zhimin Luo
126
57
0
26 Feb 2024
Radar-Based Recognition of Static Hand Gestures in American Sign
  Language
Radar-Based Recognition of Static Hand Gestures in American Sign Language
C. Schuessler
Wenxuan Zhang
Johanna Braunig
Marcel Hoffmann
Michael Stelzig
Martin Vossiek
32
3
0
20 Feb 2024
DeepATLAS: One-Shot Localization for Biomedical Data
DeepATLAS: One-Shot Localization for Biomedical Data
Peter D. Chang
49
0
0
14 Feb 2024
Switch EMA: A Free Lunch for Better Flatness and Sharpness
Switch EMA: A Free Lunch for Better Flatness and Sharpness
Siyuan Li
Zicheng Liu
Juanxi Tian
Ge Wang
Zedong Wang
...
Cheng Tan
Tao Lin
Yang Liu
Baigui Sun
Stan Z. Li
66
6
0
14 Feb 2024
PFCM: Poisson flow consistency models for low-dose CT image denoising
PFCM: Poisson flow consistency models for low-dose CT image denoising
Dennis Hein
Adam Wang
Ge Wang
Ge Wang
DiffM
76
1
0
13 Feb 2024
Expert-Adaptive Medical Image Segmentation
Expert-Adaptive Medical Image Segmentation
Binyan Hu
•. A. K. Qin
63
0
0
11 Feb 2024
Two-Stage Multi-task Self-Supervised Learning for Medical Image
  Segmentation
Two-Stage Multi-task Self-Supervised Learning for Medical Image Segmentation
Binyan Hu
•. A. K. Qin
SSL
50
0
0
11 Feb 2024
Data-induced multiscale losses and efficient multirate gradient descent
  schemes
Data-induced multiscale losses and efficient multirate gradient descent schemes
Juncai He
Liangchen Liu
Yen-Hsi Tsai
90
0
0
05 Feb 2024
Balanced Resonate-and-Fire Neurons
Balanced Resonate-and-Fire Neurons
Saya Higuchi
Sebastian Kairat
S. Bohté
Sebastian Otte
52
8
0
02 Feb 2024
PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs
PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs
Ying Su
Jipeng Zhang
Yangqiu Song
Tong Zhang
62
1
0
31 Jan 2024
Inconsistency Masks: Removing the Uncertainty from Input-Pseudo-Label
  Pairs
Inconsistency Masks: Removing the Uncertainty from Input-Pseudo-Label Pairs
Michael R. H. Vorndran
Bernhard F. Roeck
VLMISegUQCV
64
3
0
25 Jan 2024
Finetuning Foundation Models for Joint Analysis Optimization
Finetuning Foundation Models for Joint Analysis Optimization
M. Vigl
N. Hartman
L. Heinrich
90
14
0
24 Jan 2024
Sequential Model for Predicting Patient Adherence in Subcutaneous
  Immunotherapy for Allergic Rhinitis
Sequential Model for Predicting Patient Adherence in Subcutaneous Immunotherapy for Allergic Rhinitis
Yin Li
Yu Xiong
Wenxin Fan
Kai Wang
Qingqing Yu
Liping Si
Patrick van der Smagt
Jun Tang
Nutan Chen
60
1
0
21 Jan 2024
Fast Registration of Photorealistic Avatars for VR Facial Animation
Fast Registration of Photorealistic Avatars for VR Facial Animation
Chaitanya Patel
Shaojie Bai
Tenia Wang
Jason M. Saragih
S. Wei
84
0
0
19 Jan 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
Kaan Ozkara
Can Karakus
Parameswaran Raman
Mingyi Hong
Shoham Sabach
Branislav Kveton
Volkan Cevher
92
4
0
17 Jan 2024
A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons
  and Adaptable Structure
A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure
Yanjie Li
Weijun Li
Lina Yu
Min Wu
Jinyi Liu
...
Xin Ning
Yugui Zhang
Baoli Lu
Jian Xu
Shuang Li
74
0
0
03 Jan 2024
Brain Tumor Segmentation Based on Deep Learning, Attention Mechanisms,
  and Energy-Based Uncertainty Prediction
Brain Tumor Segmentation Based on Deep Learning, Attention Mechanisms, and Energy-Based Uncertainty Prediction
Zachary Schwehr
Sriman Achanta
77
3
0
31 Dec 2023
SAR-RARP50: Segmentation of surgical instrumentation and Action
  Recognition on Robot-Assisted Radical Prostatectomy Challenge
SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge
Dimitrios Psychogyios
Emanuele Colleoni
Beatrice van Amsterdam
Chih-Yang Li
Shu-Yu Huang
...
Santiago Rodriguez
Juanita Puentes
Pablo Arbelaez
Omid Mohareri
Danail Stoyanov
72
28
0
31 Dec 2023
Topological Obstructions and How to Avoid Them
Topological Obstructions and How to Avoid Them
Babak Esmaeili
Robin Walters
Heiko Zimmermann
Jan-Willem van de Meent
AI4CE
63
3
0
12 Dec 2023
Eroding Trust In Aerial Imagery: Comprehensive Analysis and Evaluation
  Of Adversarial Attacks In Geospatial Systems
Eroding Trust In Aerial Imagery: Comprehensive Analysis and Evaluation Of Adversarial Attacks In Geospatial Systems
Michael Lanier
Aayush Dhakal
Zhexiao Xiong
Arthur Li
Nathan Jacobs
Yevgeniy Vorobeychik
118
0
0
12 Dec 2023
AAMDM: Accelerated Auto-regressive Motion Diffusion Model
AAMDM: Accelerated Auto-regressive Motion Diffusion Model
Tianyu Li
Calvin Qiao
Guanqiao Ren
KangKang Yin
Sehoon Ha
80
6
0
02 Dec 2023
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network
  Training
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
Yefan Zhou
Tianyu Pang
Keqin Liu
Charles H. Martin
Michael W. Mahoney
Yaoqing Yang
143
12
0
01 Dec 2023
FocusLearn: Fully-Interpretable, High-Performance Modular Neural
  Networks for Time Series
FocusLearn: Fully-Interpretable, High-Performance Modular Neural Networks for Time Series
Qiqi Su
Christos Kloukinas
Artur dÁvila Garcez
AI4TS
112
4
0
28 Nov 2023
Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large
  Catapults
Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
Prin Phunyaphibarn
Junghyun Lee
Bohan Wang
Huishuai Zhang
Chulhee Yun
80
1
0
25 Nov 2023
Expanding the deep-learning model to diagnosis LVNC: Limitations and
  trade-offs
Expanding the deep-learning model to diagnosis LVNC: Limitations and trade-offs
Gregorio Bernabé
Pilar González-Férez
J. M. García
Guillem Casas
Josefa González-Carrillo
29
3
0
23 Nov 2023
Bias-Reduced Neural Networks for Parameter Estimation in Quantitative
  MRI
Bias-Reduced Neural Networks for Parameter Estimation in Quantitative MRI
Andrew Mao
Sebastian Flassbeck
Jakob Assländer
57
2
0
13 Nov 2023
A Coefficient Makes SVRG Effective
A Coefficient Makes SVRG Effective
Yida Yin
Zhiqiu Xu
Zhiyuan Li
Trevor Darrell
Zhuang Liu
89
1
0
09 Nov 2023
Reducing Spatial Fitting Error in Distillation of Denoising Diffusion
  Models
Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models
Shengzhe Zhou
Zejian Lee
Sheng Zhang
Lefan Hou
Changyuan Yang
Guang Yang
Zhiyuan Yang
Lingyun Sun
DiffM
64
0
0
07 Nov 2023
Signal Processing Meets SGD: From Momentum to Filter
Signal Processing Meets SGD: From Momentum to Filter
Zhipeng Yao
Guisong Chang
Jiaqi Zhang
Qi Zhang
Dazhou Li
Yu Zhang
ODL
109
0
0
06 Nov 2023
Learning to Abstract with Nonparametric Variational Information
  Bottleneck
Learning to Abstract with Nonparametric Variational Information Bottleneck
Melika Behjati
Fabio Fehr
James Henderson
SSL
65
3
0
26 Oct 2023
Improving Performance in Colorectal Cancer Histology Decomposition using
  Deep and Ensemble Machine Learning
Improving Performance in Colorectal Cancer Histology Decomposition using Deep and Ensemble Machine Learning
Fabi Prezja
L. Annala
Sampsa Kiiskinen
Suvi Lahtinen
Timo Ojala
Pekka Ruusuvuori
Teijo Kuopio
74
14
0
25 Oct 2023
NuTrea: Neural Tree Search for Context-guided Multi-hop KGQA
NuTrea: Neural Tree Search for Context-guided Multi-hop KGQA
Hyeong Kyu Choi
Seunghun Lee
Jaewon Chu
Hyunwoo J. Kim
59
6
0
24 Oct 2023
Improved Techniques for Training Consistency Models
Improved Techniques for Training Consistency Models
Yang Song
Prafulla Dhariwal
85
182
0
22 Oct 2023
Decoding the Silent Majority: Inducing Belief Augmented Social Graph
  with Large Language Model for Response Forecasting
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting
Chenkai Sun
Jinning Li
Yi R. Fung
Hou Pong Chan
Tarek Abdelzaher
Chengxiang Zhai
Heng Ji
78
16
0
20 Oct 2023
Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows
Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows
David Shih
M. Freytsis
Stephen R. Taylor
J. A. Dror
Nolan Smyth
59
4
0
18 Oct 2023
Fractional Concepts in Neural Networks: Enhancing Activation Functions
Fractional Concepts in Neural Networks: Enhancing Activation Functions
Zahra Alijani
Vojtech Molek
30
1
0
18 Oct 2023
An Automatic Learning Rate Schedule Algorithm for Achieving Faster
  Convergence and Steeper Descent
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent
Zhao Song
Chiwun Yang
90
10
0
17 Oct 2023
Learning Object Permanence from Videos via Latent Imaginations
Learning Object Permanence from Videos via Latent Imaginations
Manuel Traub
Frederic Becker
S. Otte
Martin Volker Butz
57
1
0
16 Oct 2023
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete
  Representations
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations
Heyuan Yao
Zhenhua Song
Yuyang Zhou
Tenglong Ao
Baoquan Chen
Libin Liu
135
44
0
16 Oct 2023
Adam-family Methods with Decoupled Weight Decay in Deep Learning
Adam-family Methods with Decoupled Weight Decay in Deep Learning
Kuang-Yu Ding
Nachuan Xiao
Kim-Chuan Toh
64
3
0
13 Oct 2023
SSG2: A new modelling paradigm for semantic segmentation
SSG2: A new modelling paradigm for semantic segmentation
F. Diakogiannis
S. Furby
P. Caccetta
Xiaoliang Wu
Rodrigo Ibata
O. Hlinka
John Taylor
VLM
71
0
0
12 Oct 2023
Larth: Dataset and Machine Translation for Etruscan
Larth: Dataset and Machine Translation for Etruscan
Gianluca Vico
Gerasimos Spanakis
48
1
0
09 Oct 2023
Faithful Knowledge Graph Explanations for Commonsense Reasoning
Faithful Knowledge Graph Explanations for Commonsense Reasoning
Weihe Zhai
A. Zubiaga
112
0
0
07 Oct 2023
Latent Consistency Models: Synthesizing High-Resolution Images with
  Few-Step Inference
Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
Simian Luo
Yiqin Tan
Longbo Huang
Jian Li
Hang Zhao
DiffM
112
479
0
06 Oct 2023
Sparse Backpropagation for MoE Training
Sparse Backpropagation for MoE Training
Liyuan Liu
Jianfeng Gao
Weizhu Chen
MoE
68
9
0
01 Oct 2023
Small-scale proxies for large-scale Transformer training instabilities
Small-scale proxies for large-scale Transformer training instabilities
Mitchell Wortsman
Peter J. Liu
Lechao Xiao
Katie Everett
A. Alemi
...
Jascha Narain Sohl-Dickstein
Kelvin Xu
Jaehoon Lee
Justin Gilmer
Simon Kornblith
111
99
0
25 Sep 2023
Importance of Smoothness Induced by Optimizers in FL4ASR: Towards
  Understanding Federated Learning for End-to-End ASR
Importance of Smoothness Induced by Optimizers in FL4ASR: Towards Understanding Federated Learning for End-to-End ASR
Sheikh Shams Azam
Tatiana Likhomanenko
Martin Pelikan
Jan Honza Silovsky
70
7
0
22 Sep 2023
Previous
12345...161718
Next