Training Learned Optimizers with Randomly Initialized Learned Optimizers

Training Learned Optimizers with Randomly Initialized Learned Optimizers

14 January 2021

Niru Maheswaranathan

Jascha Narain Sohl-Dickstein

Papers citing "Training Learned Optimizers with Randomly Initialized Learned Optimizers"

11 / 11 papers shown

Title
Training neural networks faster with minimal tuning using pre-computed lists of hyperparameters for NAdamW Sourabh Medapati Priya Kasimbeg Shankar Krishnan Naman Agarwal George E. Dahl 51 0 0 06 Mar 2025
Investigation into the Training Dynamics of Learned Optimizers Jan Sobotka Petr Simánek Daniel Vasata 26 0 0 12 Dec 2023
Deep Model Predictive Optimization Jacob Sacks Rwik Rana Kevin Huang Alex Spitzer Guanya Shi Byron Boots 32 7 0 06 Oct 2023
Brain-inspired learning in artificial neural networks: a review Samuel Schmidgall Jascha Achterberg Thomas Miconi Louis Kirsch Rojin Ziaei S. P. Hajiseyedrazi Jason Eshraghian 16 50 0 18 May 2023
Arbitrary Order Meta-Learning with Simple Population-Based Evolution Chris Xiaoxuan Lu Sebastian Towers Jakob N. Foerster 16 5 0 16 Mar 2023
Learning to Generalize Provably in Learning to Optimize Junjie Yang Tianlong Chen Mingkang Zhu Fengxiang He Dacheng Tao Yitao Liang Zhangyang Wang 23 6 0 22 Feb 2023
Learning to Optimize with Dynamic Mode Decomposition Petr Simánek Daniel Vasata Pavel Kordík 21 5 0 29 Nov 2022
Discovering Evolution Strategies via Meta-Black-Box Optimization R. T. Lange Tom Schaul Yutian Chen Tom Zahavy Valenti Dallibard Chris Xiaoxuan Lu Satinder Singh Sebastian Flennerhag 24 47 0 21 Nov 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 21 60 0 17 Nov 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 28 32 0 22 Mar 2022
Neural Fixed-Point Acceleration for Convex Optimization Shobha Venkataraman Brandon Amos 38 13 0 21 Jul 2021