Memory-Efficient Gradient Unrolling for Large-Scale Bi-level
Optimization

Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization

20 June 2024

Yezhen Wang

Zhouhao Yang

Jonathan Scarlett

Kenji Kawaguchi

Papers citing "Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization"

14 / 14 papers shown

Title
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 34 0 0 03 May 2025
Scalable Meta-Learning via Mixed-Mode Differentiation Iurii Kemaev Dan A Calian Luisa M Zintgraf Gregory Farquhar H. V. Hasselt 52 0 0 01 May 2025
Making Scalable Meta Learning Practical Sang Keun Choe Sanket Vaibhav Mehta Hwijeen Ahn W. Neiswanger Pengtao Xie Emma Strubell Eric P. Xing 37 14 0 09 Oct 2023
Scaling Forward Gradient With Local Losses Mengye Ren Simon Kornblith Renjie Liao Geoffrey E. Hinton 61 49 0 07 Oct 2022
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach Mao Ye B. Liu S. Wright Peter Stone Qian Liu 72 82 0 19 Sep 2022
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models Adam Livska Tomávs Kovciský E. Gribovskaya Tayfun Terzi Eren Sezener ... Susannah Young Ellen Gilsenan-McMahon Sophia Austin Phil Blunsom Angeliki Lazaridou KELM 220 89 0 23 May 2022
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 222 150 0 07 Oct 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
Physics-informed neural networks with hard constraints for inverse design Lu Lu R. Pestourie Wenjie Yao Zhicheng Wang F. Verdugo Steven G. Johnson PINN 39 489 0 09 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,898 0 31 Dec 2020
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 99 714 0 13 Jun 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 237 11,568 0 09 Mar 2017
Forward and Reverse Gradient-Based Hyperparameter Optimization Luca Franceschi Michele Donini P. Frasconi Massimiliano Pontil 112 404 0 06 Mar 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,290 0 05 Nov 2016