v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Interpretable 3D Multi-Modal Residual Convolutional Neural Network for Mild Traumatic Brain Injury Diagnosis Hanem Ellethy Viktor Vegh Shekhar S. Chandra 45 4 0 22 Sep 2023
Robust Energy Consumption Prediction with a Missing Value-Resilient Metaheuristic-based Neural Network in Mobile App Development Seyed Jalaleddin Mousavirad Luís A. Alexandre 37 1 0 21 Sep 2023
EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian Ofir Gordon H. Habi Arnon Netzer MQ 73 1 0 20 Sep 2023
Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables Wenting Zhao Ye Liu Yao Wan Yibo Wang Zhongfen Deng Philip S. Yu RALM LMTD 70 7 0 20 Sep 2023
Estimating exercise-induced fatigue from thermal facial images Manuel Lage Cañellas Constantino Álvarez Casado L. Nguyen Miguel Bordallo López CVBM 48 0 0 12 Sep 2023
AdaPlus: Integrating Nesterov Momentum and Precise Stepsize Adjustment on AdamW Basis Lei Guan ODL 118 4 0 05 Sep 2023
Stochastic Variational Inference for GARCH Models Hanwen Xuan Luca Maestrini F. Chen Clara Grazian 101 2 0 29 Aug 2023
GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction Andrei Catalin Coman Christos Theodoropoulos Marie-Francine Moens James Henderson ViT 56 0 0 28 Aug 2023
Residual Denoising Diffusion Models Jiawei Liu Qiang Wang Huijie Fan Yinong Wang Yandong Tang Liangqiong Qu DiffM 130 43 0 25 Aug 2023
PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers Wenjie Zhu Peijie Qiu Xiwen Chen Oana Dumitrascu Yalin Wang 74 7 0 19 Aug 2023
Deepbet: Fast brain extraction of T1-weighted MRI using Convolutional Neural Networks L. Fisch Stefan Zumdick Carlotta B. C. Barkhau D. Emden J. Ernsting ... K. Sarink N. Winter Benjamin Risse U. Dannlowski Tim Hahn 59 6 0 14 Aug 2023
Emo-DNA: Emotion Decoupling and Alignment Learning for Cross-Corpus Speech Emotion Recognition Jiaxin Ye Yujie Wei Xin-Cheng Wen Chenglong Ma Zhizhong Huang Kunhong Liu Hongming Shan 95 2 0 04 Aug 2023
Multimodal Indoor Localisation in Parkinson's Disease for Detecting Medication Use: Observational Pilot Study in a Free-Living Setting Ferdian Jovan Catherine Morgan Ryan McConville E. Tonkin I. Craddock Alan Whone 32 3 0 03 Aug 2023
MFIM: Megapixel Facial Identity Manipulation Sanghyeon Na PICV CVBM 62 4 0 03 Aug 2023
StylePrompter: All Styles Need Is Attention Chenyi Zhuang Pan Gao A. Smolic 72 1 0 30 Jul 2023
Anatomy-Aware Lymph Node Detection in Chest CT using Implicit Station Stratification K. Yan D. Jin Dazhou Guo Minfeng Xu N. Shen Xianming Hua X. Ye Le Lu 51 5 0 28 Jul 2023
Car-Studio: Learning Car Radiance Fields from Single-View and Endless In-the-wild Images Tianyu Liu Hao Zhao Yang Yu Guyue Zhou Ming-Yuan Liu 56 3 0 26 Jul 2023
ProtoFL: Unsupervised Federated Learning via Prototypical Distillation H. Kim Youngjun Kwak Mi-Young Jung Jinho Shin Youngsung Kim Changick Kim FedML 95 10 0 23 Jul 2023
TransNet: Transparent Object Manipulation Through Category-Level Pose Estimation Huijie Zhang Anthony Opipari Xiaotong Chen Jiyue Zhu Zeren Yu Odest Chadwicke Jenkins 51 1 0 23 Jul 2023
Flatness-Aware Minimization for Domain Generalization Xingxuan Zhang Renzhe Xu Han Yu Yancheng Dong Pengfei Tian Peng Cu 87 22 0 20 Jul 2023
Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification Simon Holdenried-Krafft Peter Somers Ivonne A. Montes-Majarro Diana Silimon Cristina Tarín F. Fend Hendrik P. A. Lensch MedIm 100 3 0 14 Jul 2023
Multiplicative update rules for accelerating deep learning training and increasing robustness Manos Kirtas Nikolaos Passalis Anastasios Tefas AAML OOD 71 2 0 14 Jul 2023
Quantum Autoencoders for Learning Quantum Channel Codes Lakshika Rathi Stephen DiAdamo A. Shabani 73 3 0 13 Jul 2023
Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework Eliya Segev Maya Alroy Ronen Katsir Noam Wies Ayana Shenhav ... D. Zar Oren Tadmor Jacob Bitterman Amnon Shashua Tal Rosenwein 91 2 0 04 Jul 2023
In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification I. Dimitrovski Ivan Kitanovski Nikola Simidjievski D. Kocev SSL 59 4 0 04 Jul 2023
Relation-aware graph structure embedding with co-contrastive learning for drug-drug interaction prediction Mengying Jiang Guizhong Liu Biao Zhao Yuanchao Su Weiqiang Jin CML 95 7 0 04 Jul 2023
Neural Architecture Transfer 2: A Paradigm for Improving Efficiency in Multi-Objective Neural Architecture Search Simone Sarti Eugenio Lomurno Matteo Matteucci 51 1 0 03 Jul 2023
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers Yineng Chen Z. Li Lefei Zhang Bo Du Hai Zhao 70 4 0 02 Jul 2023
Resetting the Optimizer in Deep RL: An Empirical Study Kavosh Asadi Rasool Fakoor Shoham Sabach ODL 73 26 0 30 Jun 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 124 8 0 26 Jun 2023
Addressing Cold Start Problem for End-to-end Automatic Speech Scoring Jungbae Park Seungtaek Choi 54 5 0 25 Jun 2023
PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection Yuya Yamamoto Juhan Nam Hiroko Terasawa 19 1 0 25 Jun 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 180 314 0 24 Jun 2023
Sparse Modular Activation for Efficient Sequence Modeling Liliang Ren Yang Liu Shuohang Wang Yichong Xu Chenguang Zhu Chengxiang Zhai 95 14 0 19 Jun 2023
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference Junhao Zheng Qianli Ma Shengjie Qiu Yue Wu Peitian Ma Junlong Liu Hu Feng Xichen Shang Haibin Chen AAML KELM CML CLL 130 15 0 19 Jun 2023
Algorithms of Sampling-Frequency-Independent Layers for Non-integer Strides Kanami Imamura Tomohiko Nakamura Norihiro Takamune Kohei Yatabe Hiroshi Saruwatari 58 2 0 19 Jun 2023
Amortized Inference for Gaussian Process Hyperparameters of Structured Kernels M. Bitzer Mona Meister Christoph Zimmer 73 9 0 16 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 120 157 0 07 Jun 2023
Generalized Teacher Forcing for Learning Chaotic Dynamics Florian Hess Zahra Monfared Manuela Brenner Daniel Durstewitz AI4CE 259 36 0 07 Jun 2023
LibAUC: A Deep Learning Library for X-Risk Optimization Zhuoning Yuan Dixian Zhu Zimeng Qiu Gang Li Xuanhui Wang Tianbao Yang BDL 115 16 0 05 Jun 2023
Using Sequences of Life-events to Predict Human Lives Germans Savcisens Tina Eliassi-Rad L. K. Hansen L. Mortensen Lau Lilleholt Anna Rogers Ingo Zettler Sune Lehmann AI4TS 94 46 0 05 Jun 2023
Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on Dataset Mixtures with Uncalibrated Stereo Data Nikolay Patakin Mikhail Romanov Anna Vorontsova M. Artemyev Anton Konushin MDE 86 6 0 05 Jun 2023
End-to-End Joint Target and Non-Target Speakers ASR Ryo Masumura Naoki Makishima Taiga Yamane Yoshihiko Yamazaki Saki Mizuno ... Akihiko Takashima Satoshi Suzuki Takafumi Moriya Nobukatsu Hojo Atsushi Ando 60 5 0 04 Jun 2023
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks Dan Zhao 111 6 0 01 Jun 2023
Stochastic Gradient Langevin Dynamics Based on Quantization with Increasing Resolution Jinwuk Seok Chang-Jae Cho 50 0 0 30 May 2023
Intelligent gradient amplification for deep neural networks S. Basodi K. Pusuluri Xueli Xiao Yi Pan ODL 38 1 0 29 May 2023
CAPTDURE: Captioned Sound Dataset of Single Sources Yuki Okamoto Kanta Shimonishi Keisuke Imoto Kota Dohi Shota Horiguchi Yohei Kawaguchi 54 1 0 28 May 2023
Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTS Sewade Ogun Vincent Colotte Emmanuel Vincent DiffM 59 4 0 28 May 2023
XGrad: Boosting Gradient-Based Optimizers With Weight Prediction Lei Guan Dongsheng Li Yanqi Shi Jian Meng ODL 96 2 0 26 May 2023
Measuring the Effect of Influential Messages on Varying Personas Chenkai Sun Jinning Li Hou Pong Chan ChengXiang Zhai Heng Ji 62 6 0 25 May 2023