On the Iteration Complexity of Hypergradient Computation

29 June 2020

Luca Franceschi

Papers citing "On the Iteration Complexity of Hypergradient Computation"

50 / 127 papers shown

Title
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization Chen Fan Gaspard Choné-Ducasse Mark W. Schmidt Christos Thrampoulidis 19 3 0 30 May 2023
Effective Bilevel Optimization via Minimax Reformulation Xiaoyu Wang Rui Pan Renjie Pi Tong Zhang 35 1 0 22 May 2023
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models Dohwan Ko Joon-Young Choi Hyeong Kyu Choi Kyoung-Woon On Byungseok Roh Hyunwoo J. Kim 44 19 0 23 Mar 2023
Achieving Hierarchy-Free Approximation for Bilevel Programs With Equilibrium Constraints Jiayang Li J. Yu Boyi Liu Zhaoran Wang Y. Nie 27 6 0 20 Feb 2023
A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization Mathieu Dagréou Thomas Moreau Samuel Vaiter Pierre Ablin 23 12 0 17 Feb 2023
Hierarchical Optimization-Derived Learning Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 23 3 0 11 Feb 2023
Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation Peiyao Xiao Kaiyi Ji FedML 24 10 0 09 Feb 2023
Uncoupled Learning of Differential Stackelberg Equilibria with Commitments R. Loftin M. Çelikok H. V. Hoof Samuel Kaski F. Oliehoek 30 1 0 07 Feb 2023
Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity Risheng Liu Yaohua Liu Wei-Ting Yao Shangzhi Zeng Jin Zhang 24 24 0 07 Feb 2023
Efficient Gradient Approximation Method for Constrained Bilevel Optimization Siyuan Xu Minghui Zhu 19 19 0 03 Feb 2023
Follower Agnostic Methods for Stackelberg Games C. Maheshwari James Cheng S. S. Sasty Lillian J. Ratliff Eric Mazumdar 22 1 0 02 Feb 2023
Probabilistic Bilevel Coreset Selection Xiao Zhou Renjie Pi Weizhong Zhang Yong Lin Tong Zhang NoLa 17 27 0 24 Jan 2023
Analyzing Inexact Hypergradients for Bilevel Learning Matthias Joachim Ehrhardt Lindon Roberts 16 8 0 11 Jan 2023
First-order penalty methods for bilevel optimization Zhaosong Lu Sanyou Mei 58 31 0 04 Jan 2023
On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis Le‐Yu Chen Jing Xu J. Zhang 29 11 0 02 Jan 2023
On Implicit Bias in Overparameterized Bilevel Optimization Paul Vicol Jon Lorraine Fabian Pedregosa D. Duvenaud Roger C. Grosse AI4CE 13 36 0 28 Dec 2022
DIAMOND: Taming Sample and Communication Complexities in Decentralized Bilevel Optimization Pei-Yuan Qiu Yining Li Zhuqing Liu Prashant Khanduri Jia Liu Ness B. Shroff Elizabeth S. Bentley K. Turck 11 4 0 05 Dec 2022
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization Quan-Wu Xiao Han Shen W. Yin Tianyi Chen 16 8 0 14 Nov 2022
TorchOpt: An Efficient Library for Differentiable Optimization Jie Ren Xidong Feng Bo Liu Xuehai Pan Yao Fu Luo Mai Yaodong Yang 16 11 0 13 Nov 2022
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming Parvin Nazari Ahmad Mousavi Davoud Ataee Tarzanagh George Michailidis 19 4 0 08 Nov 2022
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models Shujian Zhang Chengyue Gong Xingchao Liu RALM 37 6 0 02 Nov 2022
Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity Xuxing Chen Minhui Huang Shiqian Ma Krishnakumar Balasubramanian 19 24 0 23 Oct 2022
How to Sift Out a Clean Data Subset in the Presence of Data Poisoning? Yi Zeng Minzhou Pan Himanshu Jahagirdar Ming Jin Lingjuan Lyu R. Jia AAML 30 21 0 12 Oct 2022
Advancing Model Pruning via Bi-level Optimization Yihua Zhang Yuguang Yao Parikshit Ram Pu Zhao Tianlong Chen Min-Fong Hong Yanzhi Wang Sijia Liu 34 68 0 08 Oct 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Di Wang Jinhui Xu 23 0 0 03 Oct 2022
The Curse of Unrolling: Rate of Differentiating Through Optimization Damien Scieur Quentin Bertrand Gauthier Gidel Fabian Pedregosa 36 11 0 27 Sep 2022
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach Mao Ye B. Liu S. Wright Peter Stone Qian Liu 72 82 0 19 Sep 2022
Differentiable Bilevel Programming for Stackelberg Congestion Games Jiayang Li J. Yu Qianni Wang Boyi Liu Zhaoran Wang Y. Nie 24 13 0 15 Sep 2022
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen-li Ma Zixuan Liu Xue Liu 81 35 0 24 Jul 2022
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods Davoud Ataee Tarzanagh Parvin Nazari Bojian Hou Li Shen Laura Balzano 37 10 0 06 Jul 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe W. Neiswanger P. Xie Eric P. Xing AI4CE 23 29 0 05 Jul 2022
On the Convergence of Distributed Stochastic Bilevel Optimization Algorithms over a Network Hongchang Gao Bin Gu My T. Thai 31 14 0 30 Jun 2022
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences Han Shen Tianyi Chen 30 15 0 21 Jun 2022
Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 29 6 0 16 Jun 2022
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems Lucy L. Gao J. Ye Haian Yin Shangzhi Zeng Jin Zhang 16 23 0 13 Jun 2022
Communication-Efficient Robust Federated Learning with Noisy Labels Junyi Li Jian Pei Heng Huang FedML 8 18 0 11 Jun 2022
Will Bilevel Optimizers Benefit from Loops Kaiyi Ji Mingrui Liu Yingbin Liang Lei Ying 33 41 0 27 May 2022
Towards Extremely Fast Bilevel Optimization with Self-governed Convergence Guarantees Risheng Liu Xuan Liu Wei-Ting Yao Shangzhi Zeng Jin Zhang 20 3 0 20 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 34 73 0 04 May 2022
A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization Ziyi Chen B. Kailkhura Yi Zhou 16 8 0 30 Mar 2022
A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima Daouda Sow Kaiyi Ji Ziwei Guan Yingbin Liang 11 8 0 01 Mar 2022
Efficiently Escaping Saddle Points in Bilevel Optimization Minhui Huang Xuxing Chen Kaiyi Ji Shiqian Ma Lifeng Lai 19 20 0 08 Feb 2022
Bilevel Optimization with a Lower-level Contraction: Optimal Sample Complexity without Warm-start Riccardo Grazzi Massimiliano Pontil Saverio Salzo 12 12 0 07 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 72 42 0 01 Feb 2022
Efficient Hyperparameter Tuning for Large Scale Kernel Ridge Regression Giacomo Meanti Luigi Carratino E. De Vito Lorenzo Rosasco 8 12 0 17 Jan 2022
A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse Junyi Li Bin Gu Heng-Chiao Huang 16 72 0 09 Dec 2021
Amortized Implicit Differentiation for Stochastic Bilevel Optimization Michael Arbel Julien Mairal 101 58 0 29 Nov 2021
Projection-Free Algorithm for Stochastic Bi-level Optimization Zeeshan Akhtar Amrit Singh Bedi Srujan Teja Thomdapu K. Rajawat 19 13 0 22 Oct 2021
On the Convergence Theory for Hessian-Free Bilevel Algorithms Daouda Sow Kaiyi Ji Yingbin Liang 22 28 0 13 Oct 2021
Value-Function-based Sequential Minimization for Bi-level Optimization Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 32 30 0 11 Oct 2021