Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 102 papers shown

Title
Argus: Federated Non-convex Bilevel Learning over 6G Space-Air-Ground Integrated Network Ya Liu Kai Yang Yu Zhu Keying Yang Haibo Zhao 21 0 0 14 May 2025
You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts Hongkun Dou Zeyu Li Xingyu Jiang Yiming Li Lijun Yang Wen Yao Yue Deng DiffM 38 0 0 12 May 2025
Frozen Layers: Memory-efficient Many-fidelity Hyperparameter Optimization Timur Carstensen Neeratyoy Mallik Frank Hutter Martin Rapp AI4CE 30 0 0 14 Apr 2025
Spectral-factorized Positive-definite Curvature Learning for NN Training Wu Lin Felix Dangel Runa Eschenhagen Juhan Bae Richard E. Turner Roger B. Grosse 47 0 0 10 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 58 3 0 31 Jan 2025
Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization Zishun Yu Tengyu Xu Di Jin Karthik Abinav Sankararaman Yun He ... Eryk Helenowski Chen Zhu Sinong Wang Hao Ma Han Fang LRM 54 4 0 29 Jan 2025
Functional Risk Minimization Ferran Alet Clement Gehring Tomás Lozano-Pérez Kenji Kawaguchi Joshua B. Tenenbaum Leslie Pack Kaelbling OffRL 60 0 0 31 Dec 2024
Influence functions and regularity tangents for efficient active learning Frederik Eaton TDI 94 0 0 22 Nov 2024
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 38 0 0 25 Oct 2024
Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios Kai Wang Zekai Li Zhi-Qi Cheng Samir Khaki A. Sajedi Ramakrishna Vedantam Konstantinos N. Plataniotis Alexander G. Hauptmann Yang You DD 70 4 0 22 Oct 2024
Differentially Private Bilevel Optimization Guy Kornowski 165 0 0 29 Sep 2024
Online Nonconvex Bilevel Optimization with Bregman Divergences Jason Bohne David Rosenberg Gary Kazantsev Pawel Polak 27 0 0 16 Sep 2024
JacNet: Learning Functions with Structured Jacobians Jonathan Lorraine Safwan Hossain 48 6 0 23 Aug 2024
FIVB ranking: Misstep in the right direction Salma Tenni Daniel Gomes de Pinho Zanco L. Szczecinski 27 0 0 02 Aug 2024
Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspective Changwen Zheng Wenwen Qiang Jianqi Zhang Changwen Zheng Jingyao Wang SSL 66 0 0 19 Jul 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 78 5 0 18 Jun 2024
Better than classical? The subtle art of benchmarking quantum machine learning models Joseph Bowles Shahnawaz Ahmed Maria Schuld 42 65 0 11 Mar 2024
Tune without Validation: Searching for Learning Rate and Weight Decay on Training Sets Lorenzo Brigato S. Mougiakakou 45 0 0 08 Mar 2024
IMMA: Immunizing text-to-image Models against Malicious Adaptation Yijia Zheng Raymond A. Yeh 50 8 0 30 Nov 2023
Farzi Data: Autoregressive Data Distillation Noveen Sachdeva Zexue He Wang-Cheng Kang Jianmo Ni D. Cheng Julian McAuley DD 23 3 0 15 Oct 2023
Doubly Robust Instance-Reweighted Adversarial Training Daouda Sow Sen-Fon Lin Zhangyang Wang Yitao Liang AAML OOD 33 2 0 01 Aug 2023
Sharpness-Aware Graph Collaborative Filtering Huiyuan Chen Chin-Chia Michael Yeh Yujie Fan Yan Zheng Junpeng Wang Vivian Lai Mahashweta Das Hao Yang 31 5 0 18 Jul 2023
Automated Polynomial Filter Learning for Graph Neural Networks Wendi Yu Zhichao Hou Xiaorui Liu 24 0 0 16 Jul 2023
Taming Small-sample Bias in Low-budget Active Learning Linxin Song Jieyu Zhang Xiaotian Lu Dinesh Manocha AI4CE 20 0 0 19 Jun 2023
Differentiable Instruction Optimization for Cross-Task Generalization Masaru Isonuma Junichiro Mori Ichiro Sakata 29 0 0 16 Jun 2023
Efficient Gradient Approximation Method for Constrained Bilevel Optimization Siyuan Xu Minghui Zhu 29 20 0 03 Feb 2023
Online Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 38 5 0 30 Jan 2023
Probabilistic Bilevel Coreset Selection Xiao Zhou Renjie Pi Weizhong Zhang Yong Lin Tong Zhang NoLa 31 27 0 24 Jan 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 39 60 0 24 Jan 2023
Dataset Distillation: A Comprehensive Review Ruonan Yu Songhua Liu Xinchao Wang DD 53 121 0 17 Jan 2023
Data Distillation: A Survey Noveen Sachdeva Julian McAuley DD 45 73 0 11 Jan 2023
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems Jérôme Bolte Edouard Pauwels Antonio Silveti-Falls 26 13 0 15 Dec 2022
Task Discovery: Finding the Tasks that Neural Networks Generalize on Andrei Atanov Andrei Filatov Teresa Yeo Ajay Sohmshetty Amir Zamir OOD 45 10 0 01 Dec 2022
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming Parvin Nazari Ahmad Mousavi Davoud Ataee Tarzanagh George Michailidis 28 4 0 08 Nov 2022
Robust Target Training for Multi-Source Domain Adaptation Zhongying Deng Da Li Yi-Zhe Song Tao Xiang OOD 24 3 0 04 Oct 2022
Task Selection for AutoML System Evaluation Jon Lorraine Nihesh Anderson Chansoo Lee Quentin de Laroussilhe Mehadi Hassen 49 4 0 26 Aug 2022
Neural network layers as parametric spans M. Bergomi Pietro Vertechi AI4CE 33 2 0 01 Aug 2022
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen Ma Zixuan Liu Xue Liu 94 35 0 24 Jul 2022
Theseus: A Library for Differentiable Nonlinear Optimization Luis Pineda Taosha Fan Maurizio Monge S. Venkataraman Paloma Sodhi ... Austin S. Wang Stuart Anderson Jing Dong Brandon Amos Mustafa Mukadam 29 76 0 19 Jul 2022
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods Davoud Ataee Tarzanagh Parvin Nazari Bojian Hou Li Shen Laura Balzano 49 10 0 06 Jul 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe W. Neiswanger P. Xie Eric P. Xing AI4CE 31 30 0 05 Jul 2022
A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks Z. Yang Richard Sinnott James Bailey Qiuhong Ke 26 39 0 14 Jun 2022
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems Lucy L. Gao J. Ye Haian Yin Shangzhi Zeng Jin Zhang 29 23 0 13 Jun 2022
Dataset Distillation using Neural Feature Regression Yongchao Zhou E. Nezhadarya Jimmy Ba DD FedML 47 151 0 01 Jun 2022
Automatic differentiation of nonsmooth iterative algorithms Jérôme Bolte Edouard Pauwels Samuel Vaiter 23 22 0 31 May 2022
Revisiting GANs by Best-Response Constraint: Perspective, Methodology, and Application Risheng Liu Jiaxin Gao Xuan Liu Xin-Yue Fan 37 4 0 20 May 2022
Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization Nir Shlezinger Yonina C. Eldar Stephen P. Boyd 30 130 0 05 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 42 73 0 04 May 2022
Machines of finite depth: towards a formalization of neural networks Pietro Vertechi M. Bergomi PINN 21 2 0 27 Apr 2022
Relaxing Equivariance Constraints with Non-stationary Continuous Filters Tycho F. A. van der Ouderaa David W. Romero Mark van der Wilk 24 33 0 14 Apr 2022