v1v2v3 (latest)

Online Learning Rate Adaptation with Hypergradient Descent

14 March 2017

Papers citing "Online Learning Rate Adaptation with Hypergradient Descent"

50 / 143 papers shown

Title
Dynamics of Learning: Generative Schedules from Latent ODEs Matt L. Sampson Peter Melchior 80 0 0 27 Sep 2025
Gradient Methods with Online Scaling Part II. Practical Aspects Ya-Chi Chu Wenzhi Gao Yinyu Ye Madeleine Udell ODL 253 1 0 13 Sep 2025
Neural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes Chengcheng Yan Jiawei Xu Zheng Peng Qingsong Wang 82 0 0 06 Aug 2025
Stress-Aware Resilient Neural Training Ashkan Shakarami Yousef Yeganeh Azade Farshad Lorenzo Nicolè Stefano Ghidoni Nassir Navab 76 1 0 31 Jul 2025
Towards Robust Learning to Optimize with Theoretical GuaranteesComputer Vision and Pattern Recognition (CVPR), 2024 Qingyu Song Wei Lin Juncheng Wang Hong Xu 132 3 0 17 Jun 2025
Gradient Methods with Online Scaling Part I. Theoretical Foundations Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 229 3 0 29 May 2025
Accelerating Optimization via Differentiable Stopping Time Zhonglin Xie Yiman Fong Haoran Yuan Zaiwen Wen 88 0 0 28 May 2025
From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients Nicolas Michel Maorong Wang Jiangpeng He Toshihiko Yamasaki CLL 184 0 0 26 Feb 2025
B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught ReasonersInternational Conference on Learning Representations (ICLR), 2024 Weihao Zeng Yuzhen Huang Lulu Zhao Yijun Wang Zifei Shan Junxian He LRM 411 22 0 23 Dec 2024
Gradient Methods with Online ScalingAnnual Conference Computational Learning Theory (COLT), 2024 Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 259 13 0 04 Nov 2024
Dynamic Estimation of Learning Rates Using a Non-Linear Autoregressive Model Ramin Okhrati 101 0 0 13 Oct 2024
Narrowing the Focus: Learned Optimizers for Pretrained Models Gus Kristiansen Mark Sandler A. Zhmoginov Nolan Miller Anirudh Goyal Jihwan Lee Max Vladymyrov 186 2 0 17 Aug 2024
Stepping on the Edge: Curvature Aware Learning Rate Tuners Vincent Roulet Atish Agarwala Jean-Bastien Grill Grzegorz Swirszcz Mathieu Blondel Fabian Pedregosa 274 4 0 08 Jul 2024
Resolving Variable Respiratory Motion From Unsorted 4D Computed Tomography Yuliang Huang Bjoern Eiben Kris Thielemans Jamie R McClelland OOD 83 3 0 30 Jun 2024
DeepMpMRI: Tensor-decomposition Regularized Learning for Fast and High-Fidelity Multi-Parametric Microstructural MR Imaging Wenxin Fan Jian Cheng Qiyuan Tian Xinrui Ma Jing Yang J. Zou Shanshan Wang MedIm 187 2 0 06 May 2024
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction Yuren Mao Xuemei Dong Wenyi Xu Yunjun Gao Bin Wei Ying Zhang 143 17 0 21 Mar 2024
Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts Sai Ashish Somayajula Youwei Liang Abhishek Singh Li Zhang Pengtao Xie 187 3 0 19 Mar 2024
A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma ... Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang 170 16 0 17 Mar 2024
Beyond Single-Model Views for Deep Learning: Optimization versus Generalizability of Stochastic Optimization Algorithms Toki Tahmid Inan Mingrui Liu Amarda Shehu 174 0 0 01 Mar 2024
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Han Guo Ramtin Hosseini Ruiyi Zhang Sai Ashish Somayajula Ranak Roy Chowdhury Rajesh K. Gupta Pengtao Xie 242 0 0 28 Feb 2024
Tuning-Free Stochastic Optimization Ahmed Khaled Chi Jin 182 13 0 12 Feb 2024
MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters Arsalan Sharifnassab Saber Salehkaleybar Richard Sutton 321 3 0 04 Feb 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient DescentInternational Conference on Machine Learning (ICML), 2024 Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach Branislav Kveton Volkan Cevher 370 6 0 17 Jan 2024
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 180 4 0 19 Dec 2023
Generating Interpretable Networks using Hypernetworks Isaac Liao Ziming Liu Max Tegmark 184 2 0 05 Dec 2023
Locally Optimal Descent for Dynamic Stepsize SchedulingInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Gilad Yehudai Alon Cohen Amit Daniely Yoel Drori Tomer Koren Mariano Schain 194 0 0 23 Nov 2023
Fast Trainable Projection for Robust Fine-TuningNeural Information Processing Systems (NeurIPS), 2023 Junjiao Tian Yen-Cheng Liu James Seale Smith Z. Kira OOD 221 18 0 29 Oct 2023
Studying K-FAC Heuristics by Viewing Adam through a Second-Order LensInternational Conference on Machine Learning (ICML), 2023 Ross M. Clarke José Miguel Hernández-Lobato 288 2 0 23 Oct 2023
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent Zhao Song Chiwun Yang 209 10 0 17 Oct 2023
FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient DescentInternational Conference on Learning Representations (ICLR), 2023 Ziyao Wang Jianyu Wang Ang Li FedML 228 6 0 04 Oct 2023
Online Sensitivity Optimization in Differentially Private LearningAAAI Conference on Artificial Intelligence (AAAI), 2023 Filippo Galli C. Palamidessi Tommaso Cucinotta 137 2 0 02 Oct 2023
Learning How to Propagate Messages in Graph Neural NetworksKnowledge Discovery and Data Mining (KDD), 2021 Teng Xiao Ruihao Zhang Xuetao Zhang Suhang Wang GNN 230 85 0 01 Oct 2023
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized ModelsNeural Information Processing Systems (NeurIPS), 2023 Leonardo Galli Holger Rauhut Mark Schmidt 178 16 0 22 Jun 2023
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional BacktrackingNeural Information Processing Systems (NeurIPS), 2023 Frederik Kunstner V. S. Portella Mark Schmidt Nick Harvey 221 11 0 05 Jun 2023
Optimal Sets and Solution Paths of ReLU NetworksInternational Conference on Machine Learning (ICML), 2023 Aaron Mishkin Mert Pilanci 302 5 0 31 May 2023
Towards Constituting Mathematical Structures for Learning to OptimizeInternational Conference on Machine Learning (ICML), 2023 Jialin Liu Xiaohan Chen Zinan Lin W. Yin HanQin Cai 237 16 0 29 May 2023
Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-SingleInternational Conference on Machine Learning (ICML), 2023 Paul Vicol Zico Kolter Kevin Swersky 172 8 0 21 Apr 2023
Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy Ramtin Hosseini Li Zhang Bhanu Garg P. Xie FaML 144 0 0 02 Apr 2023
Meta-learning approaches for few-shot learning: A survey of recent advancesACM Computing Surveys (ACM Comput. Surv.), 2023 Hassan Gharoun Fereshteh Momenifar Fang Chen Amir H. Gandomi OOD VLM 207 115 0 13 Mar 2023
Differentiable Arbitrating in Zero-sum Markov GamesAdaptive Agents and Multi-Agent Systems (AAMAS), 2023 Jing Wang Meichen Song Feng Gao Boyi Liu Zhaoran Wang Yi Wu 297 2 0 20 Feb 2023
Contrastive Learning with Consistent Representations Zihu Wang Yu Wang Hanbin Hu Peng Li CLL 261 9 0 03 Feb 2023
QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep LearningAAAI Conference on Artificial Intelligence (AAAI), 2023 Minghan Fu Fang-Xiang Wu ODL 223 10 0 01 Feb 2023
Meta-Learning Adaptive Loss Functions Christian Raymond Qi Chen Bing Xue Mengjie Zhang 194 5 0 30 Jan 2023
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 109 0 0 24 Jan 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 378 5 0 19 Jan 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 231 1 0 18 Jan 2023
End to End Generative Meta Curriculum Learning For Medical Data AugmentationInternational Conference on Information Photonics (ICIP), 2022 Meng Li Brian C. Lovell MedIm 185 4 0 20 Dec 2022
Federated Hypergradient Descent A. K. Kan FedML 150 3 0 03 Nov 2022
Selecting and Composing Learning Rate Policies for Deep Neural NetworksACM Transactions on Intelligent Systems and Technology (ACM TIST), 2022 Yanzhao Wu Ling Liu 126 32 0 24 Oct 2022
Differentiable Self-Adaptive Learning Rate Bozhou Chen Hongzhi Wang Chenmin Ba ODL 87 4 0 19 Oct 2022