Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 102 papers shown

Title
Predicting Multi-Antenna Frequency-Selective Channels via Meta-Learned Linear Filters based on Long-Short Term Channel Decomposition Sangwoo Park Osvaldo Simeone 29 4 0 23 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations Alexander Immer Tycho F. A. van der Ouderaa Gunnar Rätsch Vincent Fortuin Mark van der Wilk BDL 39 44 0 22 Feb 2022
ACORT: A Compact Object Relation Transformer for Parameter Efficient Image Captioning J. Tan Y. Tan C. Chan Joon Huang Chuah VLM ViT 26 15 0 11 Feb 2022
Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient Max W. Shen 25 18 0 10 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 75 43 0 01 Feb 2022
AutoBalance: Optimized Loss Functions for Imbalanced Data Mingchen Li Xuechen Zhang Christos Thrampoulidis Jiasi Chen Samet Oymak 19 67 0 04 Jan 2022
Efficient Automatic Differentiation of Implicit Functions C. Margossian M. Betancourt 30 2 0 28 Dec 2021
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 27 67 0 27 Dec 2021
Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning Jiachen Yang Ethan Wang Rakshit S. Trivedi T. Zhao H. Zha 30 21 0 20 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 25 26 0 16 Dec 2021
Noether Networks: Meta-Learning Useful Conserved Quantities Ferran Alet Dylan D. Doblar Allan Zhou J. Tenenbaum Kenji Kawaguchi Chelsea Finn 73 26 0 06 Dec 2021
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 61 69 0 09 Nov 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 28 9 0 20 Oct 2021
On the Convergence Theory for Hessian-Free Bilevel Algorithms Daouda Sow Kaiyi Ji Yingbin Liang 28 28 0 13 Oct 2021
Value-Function-based Sequential Minimization for Bi-level Optimization Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 40 30 0 11 Oct 2021
Online Hyperparameter Meta-Learning with Hypergradient Distillation Haebeom Lee Hayeon Lee Jaewoong Shin Eunho Yang Timothy M. Hospedales Sung Ju Hwang DD 27 2 0 06 Oct 2021
Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond Risheng Liu Yaohua Liu Shangzhi Zeng Jin Zhang 21 80 0 01 Oct 2021
Predicting Flat-Fading Channels via Meta-Learned Closed-Form Linear Filters and Equilibrium Propagation Sangwoo Park Osvaldo Simeone 42 8 0 01 Oct 2021
Data Summarization via Bilevel Optimization Zalan Borsos Mojmír Mutný Marco Tagliasacchi Andreas Krause 30 8 0 26 Sep 2021
DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture Kaichen Zhou Lanqing Hong Shuailiang Hu Fengwei Zhou Binxin Ru Jiashi Feng Zhenguo Li 59 10 0 13 Sep 2021
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 62 137 0 09 Sep 2021
Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression William T. Stephenson Zachary Frangella Madeleine Udell Tamara Broderick 22 12 0 19 Jul 2021
Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges B. Bischl Martin Binder Michel Lang Tobias Pielok Jakob Richter ... Theresa Ullmann Marc Becker A. Boulesteix Difan Deng Marius Lindauer 82 448 0 13 Jul 2021
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients Miao Zhang Steven W. Su Shirui Pan Xiaojun Chang Ehsan Abbasnejad Reza Haffari 24 68 0 21 Jun 2021
Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error Ondrej Bohdal Yongxin Yang Timothy M. Hospedales UQCV OOD 43 21 0 17 Jun 2021
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization Risheng Liu Xuan Liu Xiaoming Yuan Shangzhi Zeng Jin Zhang 15 79 0 15 Jun 2021
Proximal Optimal Transport Modeling of Population Dynamics Charlotte Bunne Laetitia Meng-Papaxanthos Andreas Krause Marco Cuturi OT 51 81 0 11 Jun 2021
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation Evgenii Nikishin Romina Abachi Rishabh Agarwal Pierre-Luc Bacon OffRL 52 34 0 06 Jun 2021
Debiasing a First-order Heuristic for Approximate Bi-level Optimization Valerii Likhosherstov Xingyou Song K. Choromanski Jared Davis Adrian Weller AI4CE 19 5 0 04 Jun 2021
SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models Zaccharie Ramzi Florian Mannel Shaojie Bai Jean-Luc Starck P. Ciuciu Thomas Moreau 29 28 0 01 Jun 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 36 13 0 29 Apr 2021
JFB: Jacobian-Free Backpropagation for Implicit Networks Samy Wu Fung Howard Heaton Qiuwei Li Daniel McKenzie Stanley Osher W. Yin FedML 35 84 0 23 Mar 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 27 93 0 02 Mar 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 59 222 0 27 Jan 2021
Convergence Properties of Stochastic Hypergradients Riccardo Grazzi Massimiliano Pontil Saverio Salzo 19 26 0 13 Nov 2020
Teaching with Commentaries Aniruddh Raghu M. Raghu Simon Kornblith David Duvenaud Geoffrey E. Hinton 17 24 0 05 Nov 2020
Dataset Meta-Learning from Kernel Ridge-Regression Timothy Nguyen Zhourung Chen Jaehoon Lee DD 36 240 0 30 Oct 2020
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians Juhan Bae Roger C. Grosse 27 24 0 26 Oct 2020
Bilevel Optimization: Convergence Analysis and Enhanced Design Kaiyi Ji Junjie Yang Yingbin Liang 28 249 0 15 Oct 2020
Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning Zhongzheng Ren Raymond A. Yeh A. Schwing 36 95 0 02 Jul 2020
On the Iteration Complexity of Hypergradient Computation Riccardo Grazzi Luca Franceschi Massimiliano Pontil Saverio Salzo 48 193 0 29 Jun 2020
Auxiliary Learning by Implicit Differentiation Aviv Navon Idan Achituve Haggai Maron Gal Chechik Ethan Fetaya 28 59 0 22 Jun 2020
Meta Approach to Data Augmentation Optimization Ryuichiro Hataya Jan Zdenek Kazuki Yoshizoe Hideki Nakayama 32 34 0 14 Jun 2020
Coresets via Bilevel Optimization for Continual Learning and Streaming Zalan Borsos Mojmír Mutný Andreas Krause CLL 38 226 0 06 Jun 2020
UFO-BLO: Unbiased First-Order Bilevel Optimization Valerii Likhosherstov Xingyou Song K. Choromanski Jared Davis Adrian Weller 32 7 0 05 Jun 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 58 1,935 0 11 Apr 2020
On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach Yuanhao Wang Guodong Zhang Jimmy Ba 33 100 0 16 Oct 2019
Modular Meta-Learning with Shrinkage Yutian Chen A. Friesen Feryal M. P. Behbahani Arnaud Doucet David Budden Matthew W. Hoffman Nando de Freitas KELM OffRL 23 35 0 12 Sep 2019
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 110 717 0 13 Jun 2018