Using a thousand optimization tasks to learn hyperparameter search
strategies

Using a thousand optimization tasks to learn hyperparameter search strategies

27 February 2020

Niru Maheswaranathan

Jascha Narain Sohl-Dickstein

Papers citing "Using a thousand optimization tasks to learn hyperparameter search strategies"

18 / 18 papers shown

Title
One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection Filippo Betello Antonio Purificato Vittoria Vineis Gabriele Tolomei Fabrizio Silvestri 41 0 0 02 May 2025
Training neural networks faster with minimal tuning using pre-computed lists of hyperparameters for NAdamW Sourabh Medapati Priya Kasimbeg Shankar Krishnan Naman Agarwal George E. Dahl 57 0 0 06 Mar 2025
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 141 0 0 22 Jan 2025
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization Herilalaina Rakotoarison Steven Adriaensen Neeratyoy Mallik Samir Garibov Eddie Bergman Frank Hutter AI4CE 32 9 0 25 Apr 2024
Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks Steven Adriaensen Herilalaina Rakotoarison Samuel G. Müller Frank Hutter BDL 31 19 0 31 Oct 2023
Deep Pipeline Embeddings for AutoML Sebastian Pineda Arango Josif Grabocka 28 2 0 23 May 2023
General-Purpose In-Context Learning by Meta-Learning Transformers Louis Kirsch James Harrison Jascha Narain Sohl-Dickstein Luke Metz 34 72 0 08 Dec 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 29 60 0 17 Nov 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 33 32 0 22 Mar 2022
HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO Katharina Eggensperger Philip Muller Neeratyoy Mallik Matthias Feurer René Sass Aaron Klein Noor H. Awad Marius Lindauer Frank Hutter 46 100 0 14 Sep 2021
Hyperparameter Transfer Across Developer Adjustments Daniel Stoll Jörg K.H. Franke Diane Wagner Simon Selg Frank Hutter 27 12 0 25 Oct 2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 22 62 0 23 Sep 2020
Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning Matthias Feurer Katharina Eggensperger Stefan Falkner Marius Lindauer Frank Hutter 35 266 0 08 Jul 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 348 0 14 Jun 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018
Emergence of Locomotion Behaviours in Rich Environments N. Heess TB Dhruva S. Sriram Jay Lemmon J. Merel ... Tom Erez Ziyun Wang S. M. Ali Eslami Martin Riedmiller David Silver 131 928 0 07 Jul 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016