A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases

22 September 2022

Papers citing "A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases"

22 / 22 papers shown

Title
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 58 0 0 22 Jan 2025
Meta-Sparsity: Learning Optimal Sparse Structures in Multi-task Networks through Meta-learning Richa Upadhyay Ronald Phlypo Rajkumar Saini Marcus Liwicki 24 0 0 21 Jan 2025
$Applications of fractional calculus in learned optimization$ Applications of fractional calculus in learned optimization Teodor Alexandru Szente James Harrison M. Zanfir C. Sminchisescu 56 0 0 22 Nov 2024
Narrowing the Focus: Learned Optimizers for Pretrained Models Gus Kristiansen Mark Sandler A. Zhmoginov Nolan Miller Anirudh Goyal Jihwan Lee Max Vladymyrov 19 1 0 17 Aug 2024
Leveraging Knowledge Distillation for Lightweight Skin Cancer Classification: Balancing Accuracy and Computational Efficiency Niful Islam Khan Md. Hasib Fahmida Akter Joti Asif Karim Sami Azam 21 0 0 24 Jun 2024
$μ$ LO: Compute-Efficient Meta-Generalization of Learned Optimizers Benjamin Thérien Charles-Étienne Joseph Boris Knyazev Edouard Oyallon Irina Rish Eugene Belilovsky AI4CE 25 1 0 31 May 2024
Graph Neural Networks for Learning Equivariant Representations of Neural Networks Miltiadis Kofinas Boris Knyazev Yan Zhang Yunlu Chen Gertjan J. Burghouts E. Gavves Cees G. M. Snoek David W. Zhang 34 29 0 18 Mar 2024
Universal Neural Functionals Allan Zhou Chelsea Finn James Harrison 17 12 0 07 Feb 2024
Investigation into the Training Dynamics of Learned Optimizers Jan Sobotka Petr Simánek Daniel Vasata 13 0 0 12 Dec 2023
Can We Learn Communication-Efficient Optimizers? Charles-Étienne Joseph Benjamin Thérien A. Moudgil Boris Knyazev Eugene Belilovsky 13 1 0 02 Dec 2023
Towards Constituting Mathematical Structures for Learning to Optimize Jialin Liu Xiaohan Chen Zhangyang Wang W. Yin HanQin Cai 14 11 0 29 May 2023
HUB: Guiding Learned Optimizers with Continuous Prompt Tuning Gaole Dai Wei Yu Wu Ziyu Wang Jie Fu Shanghang Zhang Tiejun Huang AIFin 9 0 0 26 May 2023
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies Oscar Li James Harrison Jascha Narain Sohl-Dickstein Virginia Smith Luke Metz 29 5 0 21 Apr 2023
Improving physics-informed neural networks with meta-learned optimization Alexander Bihlo PINN 21 18 0 13 Mar 2023
Learning to Optimize for Reinforcement Learning Qingfeng Lan Rupam Mahmood Shuicheng Yan Zhongwen Xu OffRL 8 4 0 03 Feb 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 27 5 0 19 Jan 2023
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 14 60 0 17 Nov 2022
Tutorial on amortized optimization Brandon Amos OffRL 63 42 0 01 Feb 2022
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping James Martens Andy Ballard Guillaume Desjardins G. Swirszcz Valentin Dalibard Jascha Narain Sohl-Dickstein S. Schoenholz 83 43 0 05 Oct 2021
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 99 714 0 13 Jun 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 237 11,568 0 09 Mar 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016