Learning Gradient Descent: Better Generalization and Longer Horizons

10 March 2017

Papers citing "Learning Gradient Descent: Better Generalization and Longer Horizons"

50 / 61 papers shown

Title
Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach Rares Cristian Pavithra Harsha Georgia Perakis Brian Quanz 12 0 0 16 May 2025
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Wenhao Li Bo Jin Mingyi Hong Changhong Lu Xiangfeng Wang 48 0 0 07 May 2025
Make Optimization Once and for All with Fine-grained Guidance Mingjia Shi Ruihan Lin Xuxi Chen Yuhao Zhou Zezhen Ding ... Tong Wang Kai Wang Zhangyang Wang Jun Zhang Tianlong Chen 55 1 0 14 Mar 2025
Meta-Sparsity: Learning Optimal Sparse Structures in Multi-task Networks through Meta-learning Richa Upadhyay Ronald Phlypo Rajkumar Saini Marcus Liwicki 40 0 0 21 Jan 2025
A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methods Wei Lin Qingyu Song Hong Xu 94 1 0 25 Nov 2024
Narrowing the Focus: Learned Optimizers for Pretrained Models Gus Kristiansen Mark Sandler A. Zhmoginov Nolan Miller Anirudh Goyal Jihwan Lee Max Vladymyrov 39 1 0 17 Aug 2024
Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learning Renzhi Wang Piji Li 37 4 0 28 May 2024
From Learning to Optimize to Learning Optimization Algorithms Camille Castera Peter Ochs 65 1 0 28 May 2024
Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Process Zhenan Fan Bissan Ghaddar Xinglu Wang Linzi Xing Yong Zhang Zirui Zhou AI4CE 53 11 0 06 Jan 2024
Investigation into the Training Dynamics of Learned Optimizers Jan Sobotka Petr Simánek Daniel Vasata 28 0 0 12 Dec 2023
Can We Learn Communication-Efficient Optimizers? Charles-Étienne Joseph Benjamin Thérien A. Moudgil Boris Knyazev Eugene Belilovsky 40 1 0 02 Dec 2023
Learning to optimize by multi-gradient for multi-objective optimization Linxi Yang Xinmin Yang L. Tang 18 1 0 01 Nov 2023
Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months Fady Rezk Antreas Antoniou Henry Gouk Timothy M. Hospedales ELM 16 1 0 27 Oct 2023
Deep Model Predictive Optimization Jacob Sacks Rwik Rana Kevin Huang Alex Spitzer Guanya Shi Byron Boots 46 7 0 06 Oct 2023
Towards Constituting Mathematical Structures for Learning to Optimize Jialin Liu Xiaohan Chen Zhangyang Wang W. Yin HanQin Cai 34 12 0 29 May 2023
Stochastic Unrolled Federated Learning Samar Hadou Navid Naderializadeh Alejandro Ribeiro FedML 30 5 0 24 May 2023
Improving physics-informed neural networks with meta-learned optimization Alexander Bihlo PINN 36 18 0 13 Mar 2023
M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation Junjie Yang Xuxi Chen Tianlong Chen Zhangyang Wang Yitao Liang 18 2 0 28 Feb 2023
Learning to Generalize Provably in Learning to Optimize Junjie Yang Tianlong Chen Mingkang Zhu Fengxiang He Dacheng Tao Yitao Liang Zhangyang Wang 31 6 0 22 Feb 2023
Learning to Optimize for Reinforcement Learning Qingfeng Lan Rupam Mahmood Shuicheng Yan Zhongwen Xu OffRL 28 6 0 03 Feb 2023
Mnemosyne: Learning to Train Transformers with Transformers Deepali Jain K. Choromanski Kumar Avinava Dubey Sumeet Singh Vikas Sindhwani Tingnan Zhang Jie Tan OffRL 39 9 0 02 Feb 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
Learning to Optimize in Model Predictive Control Jacob Sacks Byron Boots 27 22 0 05 Dec 2022
Learning to Optimize with Dynamic Mode Decomposition Petr Simánek Daniel Vasata Pavel Kordík 31 5 0 29 Nov 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 35 60 0 17 Nov 2022
Learning to Optimize Quasi-Newton Methods Isaac Liao Rumen Dangovski Jakob N. Foerster Marin Soljacic 38 4 0 11 Oct 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 75 65 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen Ma Zixuan Liu Xue Liu 94 35 0 24 Jul 2022
Automated Dynamic Algorithm Configuration Steven Adriaensen André Biedenkapp Gresa Shala Noor H. Awad Theresa Eimer Marius Lindauer Frank Hutter 32 36 0 27 May 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 38 32 0 22 Mar 2022
Symbolic Learning to Optimize: Towards Interpretability and Scalability Wenqing Zheng Tianlong Chen Ting-Kuei Hu Zhangyang Wang 45 19 0 13 Mar 2022
Optimizer Amalgamation Tianshu Huang Tianlong Chen Sijia Liu Shiyu Chang Lisa Amini Zhangyang Wang MoMe 28 4 0 12 Mar 2022
Tutorial on amortized optimization Brandon Amos OffRL 75 43 0 01 Feb 2022
A Simple Guard for Learned Optimizers Isabeau Prémont-Schwarz Jaroslav Vítkru Jan Feyereisl 49 7 0 28 Jan 2022
ModelPred: A Framework for Predicting Trained Model from Training Data Yingyan Zeng Jiachen T. Wang Si-An Chen H. Just Ran Jin R. Jia TDI MU 33 2 0 24 Nov 2021
Self-Learning Tuning for Post-Silicon Validation P. Domanski Dirk Pflüger J. Rivoir Raphael Latty 24 5 0 17 Nov 2021
Gradients are Not All You Need Luke Metz C. Freeman S. Schoenholz Tal Kachman 28 93 0 10 Nov 2021
Efficient Meta Subspace Optimization Yoni Choukroun Michael Katz 25 1 0 28 Oct 2021
Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness Xiao Yang Yinpeng Dong Wenzhao Xiang Tianyu Pang Hang Su Jun Zhu AAML 27 4 0 13 Oct 2021
Bootstrapped Meta-Learning Sebastian Flennerhag Yannick Schroecker Tom Zahavy Hado van Hasselt David Silver Satinder Singh 38 59 0 09 Sep 2021
Learn2Hop: Learned Optimization on Rough Landscapes Amil Merchant Luke Metz S. Schoenholz E. D. Cubuk 31 16 0 20 Jul 2021
Learning to Optimize: A Primer and A Benchmark Tianlong Chen Xiaohan Chen Wuyang Chen Howard Heaton Jialin Liu Zhangyang Wang W. Yin 43 225 0 23 Mar 2021
Reverse engineering learned optimizers reveals known and novel mechanisms Niru Maheswaranathan David Sussillo Luke Metz Ruoxi Sun Jascha Narain Sohl-Dickstein 22 21 0 04 Nov 2020
Training Stronger Baselines for Learning to Optimize Tianlong Chen Weiyi Zhang Jingyang Zhou Shiyu Chang Sijia Liu Lisa Amini Zhangyang Wang OffRL 25 51 0 18 Oct 2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 33 62 0 23 Sep 2020
Adaptive Hierarchical Hyper-gradient Descent Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran 21 5 0 17 Aug 2020
MLR-SNet: Transferable LR Schedules for Heterogeneous Tasks Jun Shu Yanwen Zhu Qian Zhao Zongben Xu Deyu Meng 23 7 0 29 Jul 2020
MTL2L: A Context Aware Neural Optimiser N. Kuo Mehrtash Harandi Nicolas Fourrier Christian J. Walder Gabriela Ferraro H. Suominen 12 0 0 18 Jul 2020
Guarantees for Tuning the Step Size using a Learning-to-Learn Approach Xiang Wang Shuai Yuan Chenwei Wu Rong Ge 10 16 0 30 Jun 2020