v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding Haoming Jiang Tianyu Cao Zheng Li Cheng-hsin Luo Xianfeng Tang Qingyu Yin Danqing Zhang R. Goutam Bing Yin RALM 69 12 0 08 Oct 2022
SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction Jungbae Park Jinyoung Kim Soonwoo Kwon Sang Wan Lee 34 1 0 07 Oct 2022
Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence Sung‐Jin Hong Jisu Nam Seokju Cho Susung Hong Sangryul Jeon Dongbo Min Seung Wook Kim 3DV 94 21 0 06 Oct 2022
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data Yuki Takezawa Hang Bao Kenta Niwa Ryoma Sato Makoto Yamada 76 20 0 30 Sep 2022
Automatic satellite building construction monitoring Insaf Ashrapov D. Malakhov A. Marchenkov Anton Lulin Dani El-Ayyass 20 0 0 29 Sep 2022
Multi-encoder attention-based architectures for sound recognition with partial visual assistance Wim Boes Hugo Van hamme 51 1 0 26 Sep 2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization Gábor Melis MoMe 93 1 0 26 Sep 2022
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering Chen Zheng Parisa Kordjamshidi 42 6 0 20 Sep 2022
ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech Saeed Ghorbani Ylva Ferstl Daniel Holden N. Troje M. Carbonneau 121 83 0 15 Sep 2022
Beat Transformer: Demixed Beat and Downbeat Tracking with Dilated Self-Attention Jingwei Zhao Gus Xia Ye Wang 64 19 0 15 Sep 2022
Real-world Video Anomaly Detection by Extracting Salient Features in Videos Yudai Watanabe Makoto Okabe Y. Harada Naoji Kashima AI4TS 38 5 0 14 Sep 2022
Graph Neural Networks for Low-Energy Event Classification & Reconstruction in IceCube R. Abbasi M. Ackermann J. Adams N. Aggarwal J. Aguilar ... S. Yoshida S. Yu T. Yuan Zheng Zhang P. Zhelnin 70 24 0 07 Sep 2022
Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes A. Fedorov Eloy P. T. Geenjaar Lei Wu Tristan Sylvain T. DeRamus Margaux Luck Maria B. Misiura R. Devon Hjelm Sergey Plis Vince D. Calhoun 31 3 0 07 Sep 2022
UPAR: Unified Pedestrian Attribute Recognition and Person Retrieval Andreas Specker Mickael Cormier Jürgen Beyerer CVBM 86 30 0 06 Sep 2022
Revisiting Outer Optimization in Adversarial Training Ali Dabouei Fariborz Taherkhani Sobhan Soleymani Nasser M. Nasrabadi AAML 90 4 0 02 Sep 2022
Incorporating Task-specific Concept Knowledge into Script Learning Chenkai Sun Tie Xu Chengxiang Zhai Heng Ji 70 5 0 31 Aug 2022
Pipeline-Invariant Representation Learning for Neuroimaging Xinhui Li A. Fedorov Mrinal Mathur A. Abrol Gregory Kiar Sergey Plis Vince D. Calhoun MedIm 40 1 0 27 Aug 2022
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima Hengyu Liu Qiang Fu Lun Du Tiancheng Zhang Gensitskiy Yu. Shi Han Dongmei Zhang 150 3 0 25 Aug 2022
TransNet: Category-Level Transparent Object Pose Estimation Huijie Zhang Anthony Opipari Xiaotong Chen Jiyue Zhu Zeren Yu Odest Chadwicke Jenkins ViT 53 12 0 22 Aug 2022
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 114 70 0 20 Aug 2022
How Should We Evaluate Synthesized Environmental Sounds Yuki Okamoto Keisuke Imoto Shinnosuke Takamichi Takahiro Fukumori Y. Yamashita 44 0 0 16 Aug 2022
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models Xingyu Xie Pan Zhou Huan Li Zhouchen Lin Shuicheng Yan ODL 94 169 0 13 Aug 2022
SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation Ruida Zhang Yan Di Fabian Manhardt F. Tombari Xiangyang Ji 72 37 0 13 Aug 2022
Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning J. Hu Roberto Cavicchioli Alessandro Capotondi 128 22 0 13 Aug 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation Muhammad N. ElNokrashy Amr Hendy Mohamed Maher Mohamed Afify Hany Awadalla 62 2 0 11 Aug 2022
Flexible Unsupervised Learning for Massive MIMO Subarray Hybrid Beamforming Hamed Hojatian Jérémy Nadal J. Frigon Franccois Leduc-Primeau 37 12 0 10 Aug 2022
Adaptive Learning Rates for Faster Stochastic Gradient Methods Samuel Horváth Konstantin Mishchenko Peter Richtárik ODL 63 9 0 10 Aug 2022
Continual Prune-and-Select: Class-incremental learning with specialized subnetworks Aleksandr Dekhovich David Tax M. Sluiter Miguel A. Bessa CLL 70 21 0 09 Aug 2022
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations Xufeng Zhao C. Weber Muhammad Burhan Hafez S. Wermter 62 9 0 04 Aug 2022
SGEM: stochastic gradient with energy and momentum Hailiang Liu Xuping Tian 35 4 0 03 Aug 2022
PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking? Aleksandr Kim Guillem Brasó Aljosa Osep Laura Leal-Taixé 3DPC 98 51 0 03 Aug 2022
RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation Ruida Zhang Yan Di Zhiqiang Lou Fabian Manhardt F. Tombari Xiangyang Ji 3DPC 111 48 0 30 Jul 2022
Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network Da-Rong Liu Po-Chun Hsu Yi-Chen Chen Sung-Feng Huang Shun-Po Chuang Da-Yi Wu Hung-yi Lee GAN 67 7 0 29 Jul 2022
PEA: Improving the Performance of ReLU Networks for Free by Using Progressive Ensemble Activations Á. Utasi 49 0 0 28 Jul 2022
One-Trimap Video Matting Hongje Seong Seoung Wug Oh Brian L. Price Euntai Kim Joon-Young Lee 101 13 0 27 Jul 2022
Moment Centralization based Gradient Descent Optimizers for Convolutional Neural Networks Sumanth Sadu S. Dubey S. Sreeja ODL 67 1 0 19 Jul 2022
CATRE: Iterative Point Clouds Alignment for Category-level Object Pose Refinement Xingyu Liu Gu Wang Yi Li Xiangyang Ji 3DPC 77 29 0 17 Jul 2022
Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification I. Dimitrovski Ivan Kitanovski D. Kocev Nikola Simidjievski VLM 101 78 0 14 Jul 2022
Speaker consistency loss and step-wise optimization for semi-supervised joint training of TTS and ASR using unpaired text data Naoki Makishima Satoshi Suzuki Atsushi Ando Ryo Masumura 246 5 0 11 Jul 2022
Joint Analysis of Acoustic Scenes and Sound Events with Weakly labeled Data Shunsuke Tsubaki Keisuke Imoto Nobutaka Ono 27 2 0 10 Jul 2022
Exploring the sequence length bottleneck in the Transformer for Image Captioning Jiapeng Hu Roberto Cavicchioli Alessandro Capotondi ViT 68 3 0 07 Jul 2022
A Deep Learning Approach for the solution of Probability Density Evolution of Stochastic Systems S. Pourtakdoust Amir H. Khodabakhsh 71 14 0 05 Jul 2022
TMGAN-PLC: Audio Packet Loss Concealment using Temporal Memory Generative Adversarial Network Yuansheng Guan Guochen Yu Andong Li C. Zheng Jie Wang 112 9 0 04 Jul 2022
TTS-by-TTS 2: Data-selective augmentation for neural speech synthesis using ranking support vector machine with variational autoencoder Eunwoo Song Ryuichi Yamamoto Ohsung Kwon Chan Song Min-Jae Hwang Suhyeon Oh Hyun-Wook Yoon Jin-Seob Kim Jae-Min Kim 78 7 0 30 Jun 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations Akiko Eriguchi Shufang Xie Tao Qin Hany Awadalla LRM 91 8 0 30 Jun 2022
Adversarial Multi-Task Learning for Disentangling Timbre and Pitch in Singing Voice Synthesis Tae-Woo Kim Minguk Kang Gyeong-Hoon Lee AAML 164 7 0 23 Jun 2022
Joint Analysis of Acoustic Scenes and Sound Events Based on Multitask Learning with Dynamic Weight Adaptation Kayo Nada Keisuke Imoto T. Tsuchiya 42 5 0 21 Jun 2022
A Multi-grained based Attention Network for Semi-supervised Sound Event Detection Ying Hu Xiujuan Zhu Yun Li Hao-Ming Huang Liang He 56 10 0 21 Jun 2022
TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning Jinlin Xiang Eli Shlizerman CLL 71 8 0 17 Jun 2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger Zhiqi Bu Yu Wang Sheng Zha George Karypis 134 72 0 14 Jun 2022