v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning Martin Genzel Ingo Gühring Jan Macdonald M. März 64 27 0 14 Jun 2022
Transformers are Meta-Reinforcement Learners Luckeciano C. Melo OffRL 86 50 0 14 Jun 2022
Simple Cues Lead to a Strong Multi-Object Tracker Jenny Seidenschwarz Guillem Brasó Victor Castro Serrano Ismail Elezi Laura Leal-Taixé VOT 88 54 0 09 Jun 2022
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse Lorenzo Noci Sotiris Anagnostidis Luca Biggio Antonio Orvieto Sidak Pal Singh Aurelien Lucchi 108 75 0 07 Jun 2022
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression Wanhua Li Xiaoke Huang Zheng Hua Zhu Yansong Tang Xiu Li Jie Zhou Jiwen Lu 116 34 0 06 Jun 2022
A Control Theoretic Framework for Adaptive Gradient Optimizers in Machine Learning Kushal Chakrabarti Nikhil Chopra ODL AI4CE 123 6 0 04 Jun 2022
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization Junchi Yang Xiang Li Niao He ODL 111 22 0 01 Jun 2022
VectorAdam for Rotation Equivariant Geometry Optimization Selena Ling Nicholas Sharp Alec Jacobson 47 16 0 26 May 2022
Learning What and Where: Disentangling Location and Identity Tracking Without Supervision Manuel Traub S. Otte Tobias Menge Matthias Karlbauer Jannik Thummel Martin Volker Butz 115 20 0 26 May 2022
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate N. H. Phong A. Santos B. Ribeiro 75 8 0 20 May 2022
Neural Network Architecture Beyond Width and Depth Zuowei Shen Haizhao Yang Shijun Zhang 3DV MDE 114 13 0 19 May 2022
Multimodal Indoor Localisation for Measuring Mobility in Parkinson's Disease using Transformers Ferdian Jovan Ryan McConville Catherine Morgan E. Tonkin Alan Whone I. Craddock 23 1 0 12 May 2022
Unified Source-Filter GAN with Harmonic-plus-Noise Source Excitation Generation Reo Yoneyama Yi-Chiao Wu Tomoki Toda 70 14 0 12 May 2022
Individualized Risk Assessment of Preoperative Opioid Use by Interpretable Neural Network Regression Yuming Sun Jian Kang Chad Brummett Yi Li AI4CE 26 2 0 07 May 2022
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models Jinhao Jiang Kun Zhou Wayne Xin Zhao Ji-Rong Wen 75 15 0 04 May 2022
StorSeismic: A new paradigm in deep learning for seismic processing R. Harsuko T. Alkhalifah 69 38 0 30 Apr 2022
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification Marcos V. Conde Kerem Turgutlu CLIP VLM 99 99 0 29 Apr 2022
Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation Ryo Terashima Ryuichi Yamamoto Eunwoo Song Yuma Shirahata Hyun-Wook Yoon Jae-Min Kim Kentaro Tachibana 52 16 0 21 Apr 2022
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs Liang Chen Peiyi Wang Runxin Xu Tianyu Liu Zhifang Sui Baobao Chang 95 14 0 19 Apr 2022
Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information Yi Zeng Minzhou Pan H. Just Lingjuan Lyu M. Qiu R. Jia AAML 96 181 0 11 Apr 2022
ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks Xiaoxuan Lou Guowen Xu Kangjie Chen Guanlin Li Jiwei Li Tianwei Zhang OOD MQ 60 0 0 07 Apr 2022
How Information on Acoustic Scenes and Sound Events Mutually Benefits Event Detection and Scene Classification Tasks Keisuke Imoto Yuka Komatsu Shunsuke Tsubaki Tatsuya Komatsu 50 5 0 05 Apr 2022
The Group Loss++: A deeper look into group loss for deep metric learning Ismail Elezi Jenny Seidenschwarz Laurin Wagner Sebastiano Vascon Alessandro Torcinovich Marcello Pelillo Laura Leal-Taixe 71 12 0 04 Apr 2022
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image Dejia Xu Yi Ding Peihao Wang Zhiwen Fan Humphrey Shi Zhangyang Wang 106 190 0 02 Apr 2022
Learning to Deblur using Light Field Generated and Real Defocus Images Lingyan Ruan Bin Chen Jizhou Li Miuling Lam 71 72 0 01 Apr 2022
Weakly Supervised Patch Label Inference Networks for Efficient Pavement Distress Detection and Recognition in the Wild Sheng Huang Wenhao Tang Guixin Huang Luwen Huangfu Dan Yang 121 9 0 31 Mar 2022
Exploiting Explainable Metrics for Augmented SGD Mahdi S. Hosseini Mathieu Tuli Konstantinos N. Plataniotis AAML 63 3 0 31 Mar 2022
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets Junyong Lee Myeonghee Lee Sunghyun Cho Seungyong Lee SupR 68 27 0 28 Mar 2022
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range Guoqiang Zhang Kenta Niwa W. Kleijn ODL 84 2 0 24 Mar 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 48 9 0 23 Mar 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 148 32 0 22 Mar 2022
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation Gu Wang Fabian Manhardt Xingyu Liu Xiangyang Ji F. Tombari SSL 81 59 0 19 Mar 2022
ESS: Learning Event-based Semantic Segmentation from Still Images Zhaoning Sun Nico Messikommer Daniel Gehrig Davide Scaramuzza 102 83 0 18 Mar 2022
Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation Heecheol Kim Yoshiyuki Ohmura Yasuo Kuniyoshi 87 28 0 18 Mar 2022
Integrating Language Guidance into Vision-based Deep Metric Learning Karsten Roth Oriol Vinyals Zeynep Akata VLM 55 29 0 16 Mar 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training Juntang Zhuang Boqing Gong Liangzhe Yuan Huayu Chen Hartwig Adam Nicha Dvornek S. Tatikonda James Duncan Ting Liu 105 157 0 15 Mar 2022
Style Transformer for Image Inversion and Editing Xueqi Hu Qiusheng Huang Zhengyi Shi Siyuan Li Changxin Gao Li Sun Qingli Li 87 56 0 15 Mar 2022
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting Yan Di Ruida Zhang Zhiqiang Lou Fabian Manhardt Xiangyang Ji Nassir Navab F. Tombari 80 122 0 15 Mar 2022
RecursiveMix: Mixed Learning with History Lingfeng Yang Xiang Li Borui Zhao Renjie Song Jian Yang VLM 86 20 0 14 Mar 2022
Optimizer Amalgamation Tianshu Huang Tianlong Chen Sijia Liu Shiyu Chang Lisa Amini Zhangyang Wang MoMe 69 4 0 12 Mar 2022
Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control L. D. Natale B. Svetozarevic Philipp Heer Colin N. Jones OffRL AI4CE 66 6 0 10 Mar 2022
Rethinking data-driven point spread function modeling with a differentiable optical model T. Liaudat Jean-Luc Starck M. Kilbinger P. Frugier 43 12 0 09 Mar 2022
SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters Albert Mosella-Montoro Javier Ruiz-Hidalgo 3DH 184 16 0 09 Mar 2022
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation Jishnu Ray Chowdhury Debanjan Mahata Cornelia Caragea 71 2 0 09 Mar 2022
Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors F. Fervers Sebastian Bullinger C. Bodensteiner Michael Arens Rainer Stiefelhagen 42 19 0 07 Mar 2022
A Multi-Scale Time-Frequency Spectrogram Discriminator for GAN-based Non-Autoregressive TTS Haohan Guo Hui Lu Xixin Wu Helen Meng 356 7 0 02 Mar 2022
DeepNet: Scaling Transformers to 1,000 Layers Hongyu Wang Shuming Ma Li Dong Shaohan Huang Dongdong Zhang Furu Wei MoE AI4CE 136 162 0 01 Mar 2022
Echofilter: A Deep Learning Segmentation Model Improves the Automation, Standardization, and Timeliness for Post-Processing Echosounder Data in Tidal Energy Streams S. Lowe L. McGarry Jessica Douglas Jason Newport Sageev Oore Chris Whidden Daniel J. Hasselman 39 3 0 19 Feb 2022
Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer Heecheol Kim Yoshiyuki Ohmura Akihiko Nagakubo Yasuo Kuniyoshi 86 25 0 19 Feb 2022
PEg TRAnsfer Workflow recognition challenge report: Does multi-modal data improve recognition? Arnaud Huaulmé Kanako Harada Quang-Minh Nguyen Bogyu Park Seungbum Hong ... Yuto Ishikawa Yuriko Harai Satoshi Kondo M. Mitsuishi Pierre Jannin 66 9 0 11 Feb 2022