Online Learning Rate Adaptation with Hypergradient Descent

14 March 2017

Papers citing "Online Learning Rate Adaptation with Hypergradient Descent"

50 / 57 papers shown

Title
B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners Weihao Zeng Yuzhen Huang Lulu Zhao Yijun Wang Zifei Shan Junxian He LRM 55 9 0 23 Dec 2024
DeepMpMRI: Tensor-decomposition Regularized Learning for Fast and High-Fidelity Multi-Parametric Microstructural MR Imaging Wen-Jie Fan Jian Cheng Qiyuan Tian Xinrui Ma Jing Yang J. Zou Shanshan Wang MedIm 32 1 0 06 May 2024
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Han Guo Ramtin Hosseini Ruiyi Zhang Sai Ashish Somayajula Ranak Roy Chowdhury Rajesh K. Gupta Pengtao Xie 39 0 0 28 Feb 2024
Tuning-Free Stochastic Optimization Ahmed Khaled Chi Jin 32 7 0 12 Feb 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach B. Kveton V. Cevher 27 2 0 17 Jan 2024
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 42 0 0 19 Dec 2023
Learning How to Propagate Messages in Graph Neural Networks Teng Xiao Zhengyu Chen Donglin Wang Suhang Wang GNN 34 76 0 01 Oct 2023
Differentiable Arbitrating in Zero-sum Markov Games Jing Wang Meichen Song Feng Gao Boyi Liu Zhaoran Wang Yi Wu 43 2 0 20 Feb 2023
Contrastive Learning with Consistent Representations Zihu Wang Yu Wang Hanbin Hu Peng Li CLL 32 5 0 03 Feb 2023
QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning Minghan Fu Fang-Xiang Wu ODL 24 7 0 01 Feb 2023
Online Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 38 5 0 30 Jan 2023
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 8 0 0 24 Jan 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 47 5 0 19 Jan 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
Federated Hypergradient Descent A. K. Kan FedML 39 3 0 03 Nov 2022
Differentiable Self-Adaptive Learning Rate Bozhou Chen Hongzhi Wang Chenmin Ba ODL 22 4 0 19 Oct 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Di Wang Jinhui Xu 32 1 0 03 Oct 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 75 65 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 49 22 0 22 Sep 2022
Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 31 11 0 19 Sep 2022
Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction Xianghang Liu Bartlomiej Twardowski Tri Kurniawan Wijaya FedML 33 1 0 30 Aug 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe Willie Neiswanger P. Xie Eric P. Xing AI4CE 31 30 0 05 Jul 2022
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method Aaron Defazio Baoyu Zhou Lin Xiao ODL 27 5 0 14 Jun 2022
Automated Dynamic Algorithm Configuration Steven Adriaensen André Biedenkapp Gresa Shala Noor H. Awad Theresa Eimer Marius Lindauer Frank Hutter 34 36 0 27 May 2022
DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images Hongzheng Yang Cheng Chen Meirui Jiang Quande Liu Jianfeng Cao Pheng Ann Heng Qi Dou OOD 40 26 0 27 May 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 29 14 0 28 Feb 2022
Step-size Adaptation Using Exponentiated Gradient Updates Ehsan Amid Rohan Anil Christopher Fifty Manfred K. Warmuth 37 9 0 31 Jan 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation Wendi Li Xiao Yang Weiqing Liu Yingce Xia Jiang Bian DiffM AI4TS 28 51 0 11 Jan 2022
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 27 68 0 27 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 29 26 0 16 Dec 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 31 9 0 20 Oct 2021
Learning by Examples Based on Multi-level Optimization Shentong Mo P. Xie 29 0 0 22 Sep 2021
Graph Contrastive Learning Automated Yuning You Tianlong Chen Yang Shen Zhangyang Wang 24 448 0 10 Jun 2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Yuchen Jin Dinesh Manocha Liangyu Zhao Yibo Zhu Chuanxiong Guo Marco Canini Arvind Krishnamurthy 37 18 0 22 May 2021
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks S. K. Roy Mercedes Eugenia Paoletti J. Haut S. Dubey Purushottam Kar A. Plaza B. B. Chaudhuri ODL 27 18 0 21 May 2021
Robust MAML: Prioritization task buffer with adaptive learning process for model-agnostic meta-learning Thanh Nguyen Tung M. Luu T. Pham Sanzhar Rakhimkul Chang D. Yoo 22 10 0 15 Mar 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 62 223 0 27 Jan 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 44 6 0 09 Nov 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 36 0 0 02 Oct 2020
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee Junyi Li Bin Gu Heng-Chiao Huang 18 41 0 01 Sep 2020
learn2learn: A Library for Meta-Learning Research Sébastien M. R. Arnold Praateek Mahajan Debajyoti Datta Ian Bunner Konstantinos Saitas Zarkias 34 95 0 27 Aug 2020
Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees Ahnjae Shin Do Yoon Kim Joo Seong Jeong Byung-Gon Chun 20 4 0 22 Jun 2020
Balancing Training for Multilingual Neural Machine Translation Xinyi Wang Yulia Tsvetkov Graham Neubig 16 99 0 14 Apr 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 93 1,935 0 11 Apr 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 262 658 0 23 Mar 2020
Statistical Adaptive Stochastic Gradient Methods Pengchuan Zhang Hunter Lang Qiang Liu Lin Xiao ODL 15 11 0 25 Feb 2020
Optimizing Data Usage via Differentiable Rewards Xinyi Wang Hieu H. Pham Paul Michel Antonios Anastasopoulos J. Carbonell Graham Neubig 18 60 0 22 Nov 2019
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 59 70 0 09 Oct 2019
Learning an Adaptive Learning Rate Schedule Zhen Xu Andrew M. Dai Jonas Kemp Luke Metz 22 61 0 20 Sep 2019