v1v2v3 (latest)

Online Learning Rate Adaptation with Hypergradient Descent

14 March 2017

Papers citing "Online Learning Rate Adaptation with Hypergradient Descent"

50 / 143 papers shown

Title
Learning to Optimize Quasi-Newton Methods Isaac Liao Rumen Dangovski Jakob N. Foerster Marin Soljacic 211 4 0 11 Oct 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Haiyan Zhao Jinhui Xu 284 1 0 03 Oct 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 234 82 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive BiasesNeural Information Processing Systems (NeurIPS), 2022 James Harrison Luke Metz Jascha Narain Sohl-Dickstein 196 28 0 22 Sep 2022
Learning Symbolic Model-Agnostic Loss Functions via Meta-LearningIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022 Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 187 14 0 19 Sep 2022
Simple and Effective Gradient-Based Tuning of Sequence-to-Sequence Models Jared Lichtarge Chris Alberti Shankar Kumar 190 5 0 10 Sep 2022
Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction Xianghang Liu Bartlomiej Twardowski Tri Kurniawan Wijaya FedML 140 3 0 30 Aug 2022
Betty: An Automatic Differentiation Library for Multilevel OptimizationInternational Conference on Learning Representations (ICLR), 2022 Sang Keun Choe Willie Neiswanger P. Xie Eric P. Xing AI4CE 204 36 0 05 Jul 2022
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method Aaron Defazio Baoyu Zhou Lin Xiao ODL 149 7 0 14 Jun 2022
Automated Dynamic Algorithm ConfigurationJournal of Artificial Intelligence Research (JAIR), 2022 Steven Adriaensen André Biedenkapp Gresa Shala Noor H. Awad Theresa Eimer Marius Lindauer Katharina Eggensperger 241 47 0 27 May 2022
DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical ImagesIEEE Transactions on Medical Imaging (IEEE TMI), 2022 Hongzheng Yang Cheng Chen Meirui Jiang Quande Liu Jianfeng Cao Pheng Ann Heng Qi Dou OOD 123 37 0 27 May 2022
Hyper-Learning for Gradient-Based Batch Size Adaptation Calum MacLellan Feng Dong 97 0 0 17 May 2022
Learning to Accelerate by the Methods of Step-size Planning Hengshuai Yao 291 0 0 01 Apr 2022
Exploiting Explainable Metrics for Augmented SGDComputer Vision and Pattern Recognition (CVPR), 2022 Mahdi S. Hosseini Mathieu Tuli Konstantinos N. Plataniotis AAML 128 3 0 31 Mar 2022
Optimizer AmalgamationInternational Conference on Learning Representations (ICLR), 2022 Tianshu Huang Tianlong Chen Sijia Liu Shiyu Chang Lisa Amini Zinan Lin MoMe 187 5 0 12 Mar 2022
Amortized Proximal OptimizationNeural Information Processing Systems (NeurIPS), 2022 Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 277 15 0 28 Feb 2022
Step-size Adaptation Using Exponentiated Gradient Updates Ehsan Amid Rohan Anil Christopher Fifty Manfred K. Warmuth 140 9 0 31 Jan 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 166 6 0 29 Jan 2022
DDG-DA: Data Distribution Generation for Predictable Concept Drift AdaptationAAAI Conference on Artificial Intelligence (AAAI), 2022 Wendi Li Xiao Yang Yuante Li Ziheng Lu Jiang Bian DiffM AI4TS 239 70 0 11 Jan 2022
A Theoretical Understanding of Gradient Bias in Meta-Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2021 Xidong Feng Bo Liu Jie Ren Luo Mai Rui Zhu Haifeng Zhang Jun Wang Yaodong Yang 206 12 0 31 Dec 2021
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution StrategiesInternational Conference on Machine Learning (ICML), 2021 Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 199 74 0 27 Dec 2021
Meta Propagation Networks for Graph Few-shot Semi-supervised LearningAAAI Conference on Artificial Intelligence (AAAI), 2021 Kaize Ding Jianling Wang James Caverlee Huan Liu SSL 219 50 0 18 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 308 30 0 16 Dec 2021
AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop Yunfei Teng Jing Wang A. Choromańska 221 2 0 30 Nov 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 279 9 0 20 Oct 2021
Learning by Examples Based on Multi-level Optimization Shentong Mo P. Xie 129 0 0 22 Sep 2021
Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information Majid Jahani S. Rusakov Zheng Shi Peter Richtárik Michael W. Mahoney Martin Takávc ODL 142 29 0 11 Sep 2021
Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training Max Mutschler Kevin Laube A. Zell ODL 100 1 0 31 Aug 2021
Automated Learning Rate Scheduler for Large-batch Training Chiheon Kim Saehoon Kim Jongmin Kim Donghoon Lee Sungwoong Kim 114 21 0 13 Jul 2021
Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks Jun Shu Deyu Meng Zongben Xu 169 11 0 06 Jul 2021
Adaptive Learning Rate and Momentum for Training Deep Neural Networks Zhiyong Hao Yixuan Jiang Huihua Yu H. Chiang ODL 85 14 0 22 Jun 2021
Graph Contrastive Learning AutomatedInternational Conference on Machine Learning (ICML), 2021 Yuning You Tianlong Chen Yang Shen Zinan Lin 202 565 0 10 Jun 2021
BERT Learns to Teach: Knowledge Distillation with Meta LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Wangchunshu Zhou Canwen Xu Julian McAuley 211 102 0 08 Jun 2021
A Generalizable Approach to Learning Optimizers Diogo Almeida Clemens Winter Jie Tang Wojciech Zaremba AI4CE 206 33 0 02 Jun 2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the FlyInternational Conference on Learning Representations (ICLR), 2021 Yuchen Jin Wanrong Zhu Liangyu Zhao Yibo Zhu Chuanxiong Guo Marco Canini Arvind Krishnamurthy 166 23 0 22 May 2021
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks S. K. Roy Mercedes Eugenia Paoletti J. Haut S. Dubey Purushottam Kar A. Plaza B. B. Chaudhuri ODL 191 20 0 21 May 2021
Meta-Regularization: An Approach to Adaptive Choice of the Learning Rate in Gradient Descent Guangzeng Xie Hao Jin Dachao Lin Zhihua Zhang 115 0 0 12 Apr 2021
Robust MAML: Prioritization task buffer with adaptive learning process for model-agnostic meta-learningIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2021 Thanh Nguyen Tung M. Luu T. Pham Sanzhar Rakhimkul Chang D. Yoo 110 16 0 15 Mar 2021
Population-Based Evolution Optimizes a Meta-Learning Objective Kevin Frans Olaf Witkowski 178 6 0 11 Mar 2021
A Probabilistically Motivated Learning Rate Adaptation for Stochastic Optimization Filip de Roos Carl Jidling A. Wills Thomas B. Schon Philipp Hennig 92 3 0 22 Feb 2021
Meta Back-translationInternational Conference on Learning Representations (ICLR), 2021 Hieu H. Pham Xinyi Wang Yiming Yang Graham Neubig 140 26 0 15 Feb 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and BeyondIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021 Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 408 257 0 27 Jan 2021
Joint Search of Data Augmentation Policies and Network Architectures Taiga Kashima Yoshihiro Yamada Shunta Saito 3DPC 171 5 0 17 Dec 2020
Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation Gengwei Zhang Yiming Gao Hang Xu Hao Zhang Zhenguo Li Xiaodan Liang SSeg 291 5 0 07 Dec 2020
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 167 6 0 09 Nov 2020
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment Zirui Wang Zachary Chase Lipton Yulia Tsvetkov 155 32 0 06 Oct 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 169 0 0 02 Oct 2020
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee Junyi Li Bin Gu Heng-Chiao Huang 152 42 0 01 Sep 2020
learn2learn: A Library for Meta-Learning Research Sébastien M. R. Arnold Praateek Mahajan Debajyoti Datta Ian Bunner Konstantinos Saitas Zarkias 261 105 0 27 Aug 2020
Adaptive Hierarchical Hyper-gradient Descent Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran 102 5 0 17 Aug 2020