Learning-Rate-Free Learning by D-Adaptation

18 January 2023

Papers citing "Learning-Rate-Free Learning by D-Adaptation"

50 / 65 papers shown

Title
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation Robert M. Gower Guillaume Garrigos Nicolas Loizou Dimitris Oikonomou Konstantin Mishchenko Fabian Schaipp 31 0 0 02 Apr 2025
Benefits of Learning Rate Annealing for Tuning-Robustness in Stochastic Optimization Amit Attia Tomer Koren 65 1 0 13 Mar 2025
Towards hyperparameter-free optimization with differential privacy Zhiqi Bu Ruixuan Liu 24 1 0 02 Mar 2025
Adaptive Accelerated Proximal Gradient Methods with Variance Reduction for Composite Nonconvex Finite-Sum Minimization Ganzhao Yuan 35 0 0 28 Feb 2025
LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data Peer Nagy Sascha Frey Kang Li Bidipta Sarkar Svitlana Vyetrenko Stefan Zohren Ani Calinescu Jakob Foerster 84 1 0 13 Feb 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian J. Barnett 39 1 0 12 Jan 2025
Temporal Context Consistency Above All: Enhancing Long-Term Anticipation by Learning and Enforcing Temporal Constraints Alberto Maté Mariella Dimiccoli AI4TS 26 0 0 27 Dec 2024
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 77 1 0 22 Dec 2024
An Energy-Based Self-Adaptive Learning Rate for Stochastic Gradient Descent: Enhancing Unconstrained Optimization with VAV method Jiahao Zhang Christian Moya Guang Lin 26 0 0 10 Nov 2024
CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks Miria Feng Zachary Frangella Mert Pilanci BDL 38 1 0 02 Nov 2024
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang V. Cevher Mingyi Hong MU 83 6 0 29 Oct 2024
Tuning-free coreset Markov chain Monte Carlo Naitong Chen Jonathan H. Huggins Trevor Campbell 25 0 0 24 Oct 2024
A second-order-like optimizer with adaptive gradient scaling for deep learning Jérôme Bolte Ryan Boustany Edouard Pauwels Andrei Purica ODL 30 0 0 08 Oct 2024
Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning Lequan Lin Dai Shi Andi Han Zhiyong Wang Junbin Gao 23 0 0 08 Oct 2024
Old Optimizer, New Norm: An Anthology Jeremy Bernstein Laker Newhouse ODL 36 12 0 30 Sep 2024
State-free Reinforcement Learning Mingyu Chen Aldo Pacchiano Xuezhou Zhang 61 0 0 27 Sep 2024
VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos Yan-Bo Lin Yu Tian L. Yang Gedas Bertasius Heng Wang VGen 34 7 0 11 Sep 2024
Exploring Foundation Models for Synthetic Medical Imaging: A Study on Chest X-Rays and Fine-Tuning Techniques Davide Clode da Silva Marina Musse Bernardes Nathalia Giacomini Ceretta Gabriel Vaz de Souza Gabriel Fonseca Silva Rafael Heitor Bordini S. Musse MedIm LM&MA 23 0 0 06 Sep 2024
Stepping on the Edge: Curvature Aware Learning Rate Tuners Vincent Roulet Atish Agarwala Jean-Bastien Grill Grzegorz Swirszcz Mathieu Blondel Fabian Pedregosa 34 1 0 08 Jul 2024
An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes Antonio Orvieto Lin Xiao 37 2 0 05 Jul 2024
Dreamguider: Improved Training free Diffusion-based Conditional Generation Nithin Gopalakrishnan Nair Vishal M. Patel 30 2 0 04 Jun 2024
Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds Daniel Dodd Louis Sharrock Christopher Nemeth 36 0 0 04 Jun 2024
Fully Unconstrained Online Learning Ashok Cutkosky Zakaria Mhammedi CLL 27 1 0 30 May 2024
The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms Elizabeth Collins-Woodfin Inbar Seroussi Begona García Malaxechebarría Andrew W. Mackenzie Elliot Paquette Courtney Paquette 28 1 0 30 May 2024
The Road Less Scheduled Aaron Defazio Xingyu Yang Yang Harsh Mehta Konstantin Mishchenko Ahmed Khaled Ashok Cutkosky 28 45 0 24 May 2024
Scalable Optimization in the Modular Norm Tim Large Yang Liu Minyoung Huh Hyojin Bahng Phillip Isola Jeremy Bernstein 39 12 0 23 May 2024
Unleash Graph Neural Networks from Heavy Tuning Lequan Lin Dai Shi Andi Han Zhiyong Wang Junbin Gao AI4CE 27 2 0 21 May 2024
Towards Stability of Parameter-free Optimization Yijiang Pang Shuyang Yu Hoang Bao Jiayu Zhou 29 1 0 07 May 2024
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad Sayantan Choudhury N. Tupitsa Nicolas Loizou Samuel Horváth Martin Takáč Eduard A. Gorbunov 30 1 0 05 Mar 2024
Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning Wuyang Chen Jialin Song Pu Ren Shashank Subramanian Dmitriy Morozov Michael W. Mahoney AI4CE 35 9 0 24 Feb 2024
Tuning-Free Stochastic Optimization Ahmed Khaled Chi Jin 30 7 0 12 Feb 2024
AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size P. Ostroukhov Aigerim Zhumabayeva Chulu Xiang Alexander Gasnikov Martin Takáč Dmitry Kamzolov ODL 38 2 0 07 Feb 2024
How Free is Parameter-Free Stochastic Optimization? Amit Attia Tomer Koren ODL 39 4 0 05 Feb 2024
MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters Arsalan Sharifnassab Saber Salehkaleybar Richard Sutton 27 3 0 04 Feb 2024
Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence Yichuan Deng Zhao-quan Song Chiwun Yang 24 1 0 02 Feb 2024
Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity Wenzhi Gao Qi Deng 22 1 0 25 Jan 2024
Masked Audio Generation using a Single Non-Autoregressive Transformer Alon Ziv Itai Gat Gaël Le Lan Tal Remez Felix Kreuk Alexandre Défossez Jade Copet Gabriel Synnaeve Yossi Adi 40 36 0 09 Jan 2024
Interpreting Adaptive Gradient Methods by Parameter Scaling for Learning-Rate-Free Optimization Min-Kook Suh Seung-Woo Seo ODL 27 0 0 06 Jan 2024
Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera Jiye Lee Hanbyul Joo 26 16 0 01 Jan 2024
SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms Farshed Abdukhakimov Chulu Xiang Dmitry Kamzolov Robert Mansel Gower Martin Takáč 32 2 0 28 Dec 2023
Locally Optimal Descent for Dynamic Stepsize Scheduling Gilad Yehudai Alon Cohen Amit Daniely Yoel Drori Tomer Koren Mariano Schain 24 0 0 23 Nov 2023
Non-Uniform Smoothness for Gradient Descent A. Berahas Lindon Roberts Fred Roosta 20 3 0 15 Nov 2023
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent Zhao-quan Song Chiwun Yang 19 9 0 17 Oct 2023
A simple uniformly optimal method without line search for convex optimization Tianjiao Li Guanghui Lan 19 20 0 16 Oct 2023
Multiple Physics Pretraining for Physical Surrogate Models Michael McCabe Bruno Régaldo-Saint Blancard Liam Parker Ruben Ohana M. Cranmer ... Francois Lanusse Mariel Pettee Tiberiu Teşileanu Kyunghyun Cho Shirley Ho PINN AI4CE 23 51 0 04 Oct 2023
Small-scale proxies for large-scale Transformer training instabilities Mitchell Wortsman Peter J. Liu Lechao Xiao Katie Everett A. Alemi ... Jascha Narain Sohl-Dickstein Kelvin Xu Jaehoon Lee Justin Gilmer Simon Kornblith 35 80 0 25 Sep 2023
Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views Taeho Kang Kyungjin Lee Jinrui Zhang Youngki Lee EgoV 17 18 0 21 Sep 2023
Learning-Rate-Free Learning: Dissecting D-Adaptation and Probabilistic Line Search Max McGuinness ODL 13 0 0 06 Aug 2023
Adaptive Proximal Gradient Method for Convex Optimization Yura Malitsky Konstantin Mishchenko 16 21 0 04 Aug 2023
An Oblivious Stochastic Composite Optimization Algorithm for Eigenvalue Optimization Problems Clément Lezane Cristóbal Guzmán Alexandre d’Aspremont 28 0 0 30 Jun 2023