v1v2v3 (latest)

Online Learning Rate Adaptation with Hypergradient Descent

14 March 2017

Papers citing "Online Learning Rate Adaptation with Hypergradient Descent"

43 / 143 papers shown

Title
MLR-SNet: Transferable LR Schedules for Heterogeneous TasksIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 Jun Shu Yanwen Zhu Qian Zhao Zongben Xu Deyu Meng 267 8 0 29 Jul 2020
La-MAML: Look-ahead Meta Learning for Continual LearningNeural Information Processing Systems (NeurIPS), 2020 Gunshi Gupta Karmesh Yadav Liam Paull CLL VLM 271 80 0 27 Jul 2020
Gradient-based Hyperparameter Optimization Over Long HorizonsNeural Information Processing Systems (NeurIPS), 2020 P. Micaelli Amos Storkey 253 17 0 15 Jul 2020
Automatic Tuning of Stochastic Gradient Descent with Bayesian Optimisation Victor Picheny Vincent Dutordoir A. Artemev N. Durrande 108 2 0 25 Jun 2020
Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees Ahnjae Shin Do Yoon Kim Joo Seong Jeong Byung-Gon Chun 132 5 0 22 Jun 2020
AdaS: Adaptive Scheduling of Stochastic Gradients Mahdi S. Hosseini Konstantinos N. Plataniotis ODL 126 12 0 11 Jun 2020
AutoHAS: Efficient Hyperparameter and Architecture Search Xuanyi Dong Mingxing Tan Adams Wei Yu Daiyi Peng Bogdan Gabrys Quoc V. Le TPM 146 24 0 05 Jun 2020
Generalized Reinforcement Meta Learning for Few-Shot Optimization R. Anantha S. Pulman Srinivas Chappidi OffRL 82 3 0 04 May 2020
Balancing Training for Multilingual Neural Machine TranslationAnnual Meeting of the Association for Computational Linguistics (ACL), 2020 Xinyi Wang Yulia Tsvetkov Graham Neubig 372 110 0 14 Apr 2020
Meta-Learning in Neural Networks: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 641 2,334 0 11 Apr 2020
Meta Pseudo LabelsComputer Vision and Pattern Recognition (CVPR), 2020 Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 904 727 0 23 Mar 2020
Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?International Conference on Pattern Recognition (ICPR), 2020 Xiang Deng Zhongfei Zhang AI4CE 94 5 0 27 Feb 2020
Statistical Adaptive Stochastic Gradient Methods Pengchuan Zhang Hunter Lang Qiang Liu Lin Xiao ODL 167 12 0 25 Feb 2020
Stage-based Hyper-parameter Optimization for Deep Learning Ahnjae Shin Dongjin Shin Sungwoo Cho Do Yoon Kim Eunji Jeong Gyeong-In Yu Byung-Gon Chun 80 4 0 24 Nov 2019
Optimizing Data Usage via Differentiable RewardsInternational Conference on Machine Learning (ICML), 2019 Xinyi Wang Hieu H. Pham Paul Michel Antonios Anastasopoulos J. Carbonell Graham Neubig 339 66 0 22 Nov 2019
A Simple Dynamic Learning Rate Tuning Algorithm For Automated Training of DNNs Koyel Mukherjee Alind Khare Ashish Verma 143 20 0 25 Oct 2019
MARTHE: Scheduling the Learning Rate Via Online Hypergradients Michele Donini Luca Franceschi Massimiliano Pontil Orchid Majumder P. Frasconi 109 7 0 18 Oct 2019
First-Order Preconditioning via Hypergradient Descent Theodore H. Moskovitz Rui Wang Janice Lan Sanyam Kapoor Thomas Miconi J. Yosinski Aditya Rawal AI4CE 150 10 0 18 Oct 2019
On the adequacy of untuned warmup for adaptive optimizationAAAI Conference on Artificial Intelligence (AAAI), 2019 Jerry Ma Denis Yarats 282 79 0 09 Oct 2019
Gradient Descent: The Ultimate OptimizerNeural Information Processing Systems (NeurIPS), 2019 Kartik Chandra Audrey Xie Jonathan Ragan-Kelley E. Meijer ODL 154 0 0 29 Sep 2019
Using Statistics to Automate Stochastic OptimizationNeural Information Processing Systems (NeurIPS), 2019 Hunter Lang Pengchuan Zhang Lin Xiao 128 24 0 21 Sep 2019
Learning an Adaptive Learning Rate Schedule Zhen Xu Andrew M. Dai Jonas Kemp Luke Metz 115 71 0 20 Sep 2019
Adaptive Scheduling for Multi-Task Learning Sébastien Jean Orhan Firat Melvin Johnson 140 46 0 13 Sep 2019
Meta-descent for Online, Continual PredictionAAAI Conference on Artificial Intelligence (AAAI), 2019 Andrew Jacobsen M. Schlegel Cam Linke T. Degris Adam White Martha White 165 24 0 17 Jul 2019
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges N. Arivazhagan Ankur Bapna Orhan Firat Dmitry Lepikhin Melvin Johnson ... George F. Foster Colin Cherry Wolfgang Macherey Zhiwen Chen Yonghui Wu 211 447 0 11 Jul 2019
Accelerating Mini-batch SARAH by Step Size RulesInformation Sciences (Inf. Sci.), 2019 Zhuang Yang Zengping Chen Cheng-Yu Wang 194 15 0 20 Jun 2019
Training Neural Networks for and by InterpolationInternational Conference on Machine Learning (ICML), 2019 Leonard Berrada Andrew Zisserman M. P. Kumar 3DH 161 68 0 13 Jun 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence RatesNeural Information Processing Systems (NeurIPS), 2019 Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Damien Scieur ODL 360 226 0 24 May 2019
Alpha MAML: Adaptive Model-Agnostic Meta-Learning Harkirat Singh Behl A. G. Baydin Juil Sock 148 70 0 17 May 2019
Parabolic Approximation Line Search for DNNs Max Mutschler A. Zell ODL 255 20 0 28 Mar 2019
Block stochastic gradient descent for large-scale tomographic reconstruction in a parallel network Yushan Gao A. Biguri T. Blumensath 152 3 0 28 Mar 2019
Online Meta-Learning Chelsea Finn Aravind Rajeswaran Sham Kakade Sergey Levine CLL 225 288 0 22 Feb 2019
Meta-CurvatureNeural Information Processing Systems (NeurIPS), 2019 Eunbyung Park Junier B. Oliva BDL 333 132 0 09 Feb 2019
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization Zhenxun Zhuang Ashok Cutkosky Francesco Orabona 246 5 0 25 Jan 2019
Deep Frank-Wolfe For Neural Network OptimizationInternational Conference on Learning Representations (ICLR), 2018 Leonard Berrada Andrew Zisserman M. P. Kumar ODL 167 41 0 19 Nov 2018
Truncated Back-propagation for Bilevel Optimization Amirreza Shaban Ching-An Cheng Nathan Hatch Byron Boots 314 296 0 25 Oct 2018
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 341 144 0 16 Oct 2018
Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives George Tucker Dieterich Lawson S. Gu Chris J. Maddison BDL 205 111 0 09 Oct 2018
Learning with Random Learning Rates Léonard Blier Pierre Wolinski Yann Ollivier OOD 233 21 0 02 Oct 2018
Guided evolutionary strategies: Augmenting random search with surrogate gradients Niru Maheswaranathan Luke Metz George Tucker Dami Choi Jascha Narain Sohl-Dickstein 233 22 0 26 Jun 2018
L4: Practical loss-based stepsize adaptation for deep learning Michal Rolínek Georg Martius ODL 270 65 0 14 Feb 2018
Online Learning of a Memory for Learning Rates Franziska Meier Daniel Kappler S. Schaal 96 21 0 20 Sep 2017
Automatic differentiation in machine learning: a survey A. G. Baydin Barak A. Pearlmutter Alexey Radul J. Siskind PINN AI4CE ODL 434 3,215 0 20 Feb 2015