Gradient-based Hyperparameter Optimization through Reversible Learning

11 February 2015

Papers citing "Gradient-based Hyperparameter Optimization through Reversible Learning"

50 / 498 papers shown

Title
Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization Youran Dong Junfeng Yang Wei-Ting Yao Jin Zhang 239 0 0 04 May 2025
Scalable Meta-Learning via Mixed-Mode Differentiation Iurii Kemaev Dan A Calian Luisa M Zintgraf Gregory Farquhar H. V. Hasselt 57 0 0 01 May 2025
Tuning Learning Rates with the Cumulative-Learning Constant Nathan Faraj 7 0 0 30 Apr 2025
MAGIC: Near-Optimal Data Attribution for Deep Learning Andrew Ilyas Logan Engstrom TDI 48 0 0 23 Apr 2025
An experimental survey and Perspective View on Meta-Learning for Automated Algorithms Selection and Parametrization Moncef Garouani 28 0 0 08 Apr 2025
Exploring the Potential of Bilevel Optimization for Calibrating Neural Networks Gabriele Sanguin Arjun Pakrashi Marco Viola Francesco Rinaldi 60 0 0 17 Mar 2025
PRDP: Progressively Refined Differentiable Physics Kanishk Bhatia Felix Koehler Nils Thuerey AI4CE 72 0 0 26 Feb 2025
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy Chen Wang Kaiyi Ji Junyi Geng Zhongqiang Ren Taimeng Fu ... Yi Du Qihang Li Yue Yang Xiao Lin Zhipeng Zhao SSL 97 9 0 28 Jan 2025
Reinforcement Teaching Alex Lewandowski Calarina Muslimani Dale Schuurmans Matthew E. Taylor Jun Luo 87 1 0 28 Jan 2025
Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs Amirmohammad Farzaneh Osvaldo Simeone 94 0 0 22 Jan 2025
On Penalty-based Bilevel Gradient Descent Method Han Shen Quan-Wu Xiao Tianyi Chen 62 53 0 08 Jan 2025
Memory-Reduced Meta-Learning with Guaranteed Convergence Honglin Yang Ji Ma Xiao Yu 79 0 0 16 Dec 2024
A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methods Wei Lin Qingyu Song Hong Xu 103 0 0 25 Nov 2024
Zero-Shot Coreset Selection: Efficient Pruning for Unlabeled Data Brent A. Griffin Jacob Marks Jason J. Corso VLM 81 2 0 22 Nov 2024
Influence functions and regularity tangents for efficient active learning Frederik Eaton TDI 98 0 0 22 Nov 2024
Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling Junyi Li Heng Huang 39 1 0 07 Nov 2024
Reducing Hyperparameter Tuning Costs in ML, Vision and Language Model Training Pipelines via Memoization-Awareness Abdelmajid Essofi Ridwan Salahuddeen Munachiso Nwadike Elnura Zhalieva Kun Zhang Eric P. Xing Willie Neiswanger Qirong Ho VLM 49 0 0 06 Nov 2024
Gradient Methods with Online Scaling Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 41 1 0 04 Nov 2024
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 42 0 0 25 Oct 2024
Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level Optimization Shancong Mou Raviteja Vemulapalli Shiyu Li Yuxuan Liu C Thomas ... Haoping Bai Oncel Tuzel Ping Huang Jiulong Shan Jianjun Shi 47 0 0 24 Oct 2024
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates Sheheryar Mehmood Peter Ochs 31 1 0 21 Oct 2024
Optimizing importance weighting in the presence of sub-population shifts Floris Holstege Bram Wouters Noud van Giersbergen C. Diks 36 0 0 18 Oct 2024
Towards Differentiable Multilevel Optimization: A Gradient-Based Approach Yuntian Gu Xuzheng Chen 23 0 0 15 Oct 2024
Differentially Private Bilevel Optimization Guy Kornowski 216 0 0 29 Sep 2024
Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selection Matteo Zecchin Sangwoo Park Osvaldo Simeone LM&MA 79 3 0 24 Sep 2024
Online Nonconvex Bilevel Optimization with Bregman Divergences Jason Bohne David Rosenberg Gary Kazantsev Pawel Polak 40 0 0 16 Sep 2024
A New First-Order Meta-Learning Algorithm with Convergence Guarantees El Mahdi Chayti Martin Jaggi 38 1 0 05 Sep 2024
UDD: Dataset Distillation via Mining Underutilized Regions Shiguang Wang Zhongyu Zhang Jian Cheng DD 38 0 0 29 Aug 2024
Flexora: Flexible Low Rank Adaptation for Large Language Models Chenxing Wei Yao Shu Ying Tiffany He Fei Richard Yu AI4CE 36 3 0 20 Aug 2024
Narrowing the Focus: Learned Optimizers for Pretrained Models Gus Kristiansen Mark Sandler A. Zhmoginov Nolan Miller Anirudh Goyal Jihwan Lee Max Vladymyrov 41 1 0 17 Aug 2024
Generative Dataset Distillation Based on Diffusion Model Duo Su Junjie Hou Guang Li Ren Togo Rui Song Takahiro Ogawa Miki Haseyama VGen DD 43 4 0 16 Aug 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 75 93 0 05 Jul 2024
A Linear Programming Enhanced Genetic Algorithm for Hyperparameter Tuning in Machine Learning Ankur Sinha Paritosh Pankaj 41 0 0 30 Jun 2024
ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting Rui Pan Jipeng Zhang Xingyuan Pan Renjie Pi Xiaoyu Wang Tong Zhang 60 5 0 28 Jun 2024
Improving Hyperparameter Optimization with Checkpointed Model Weights Nikhil Mehta Jonathan Lorraine Steve Masson Ramanathan Arunachalam Zaid Pervaiz Bhat James Lucas Arun George Zachariah 60 4 0 26 Jun 2024
Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization Qianli Shen Yezhen Wang Zhouhao Yang Xiang Li Haonan Wang Yang Zhang Jonathan Scarlett Zhanxing Zhu Kenji Kawaguchi AI4CE 76 4 0 20 Jun 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 82 5 0 18 Jun 2024
A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints Liuyuan Jiang Quan-Wu Xiao Victor M. Tenorio Fernando Real-Rojas Antonio G. Marques Tianyi Chen 57 2 0 14 Jun 2024
Meta-Learning Neural Procedural Biases Christian Raymond Qi Chen Bing Xue Mengjie Zhan 54 1 0 12 Jun 2024
Beyond Trend Following: Deep Learning for Market Trend Prediction Fernando Berzal Alberto Garcia 55 0 0 10 Jun 2024
Derivatives of Stochastic Gradient Descent F. Iutzeler Edouard Pauwels Samuel Vaiter 42 1 0 24 May 2024
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization Chris Junchi Li 59 0 0 01 May 2024
LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace Bin Gao Yan Yang Ya-xiang Yuan 46 2 0 04 Apr 2024
DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation Aru Maekawa Satoshi Kosugi Kotaro Funakoshi Manabu Okumura DD 59 10 0 30 Mar 2024
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates Riccardo Grazzi Massimiliano Pontil Saverio Salzo 49 1 0 18 Mar 2024
AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning Ruiyi Zhang Rushi Qiang Sai Ashish Somayajula Pengtao Xie 42 13 0 14 Mar 2024
One Category One Prompt: Dataset Distillation using Diffusion Models Ali Abbasi Ashkan Shahbazi Hamed Pirsiavash Soheil Kolouri DiffM DD 42 3 0 11 Mar 2024
Fast and Efficient Local Search for Genetic Programming Based Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 51 2 0 01 Mar 2024
Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF Han Shen Zhuoran Yang Tianyi Chen OffRL 45 14 0 10 Feb 2024
Glocal Hypergradient Estimation with Koopman Operator Ryuichiro Hataya Yoshinobu Kawahara 44 2 0 05 Feb 2024