Optimization for Deep Learning

ODL

Develops efficient optimization algorithms for training deep models. Improves convergence speed and model performance.

Neighbor communities

51015

Featured Papers

0 / 0 papers shown

Title

All papers

50 / 1,060 papers shown

Title
Gradient-Variation Online Adaptivity for Accelerated Optimization with Hölder Smoothness Yuheng Zhao Yu-Hu Yan Kfir Yehuda Levy Peng Zhao ODL 72 2 0 04 Nov 2025
Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks Alexander Heinlein Taniya Kapoor ODL 76 0 0 30 Oct 2025
On the Stability of Neural Networks in Deep Learning Blaise Delattre ODL AAML 127 0 0 29 Oct 2025
How do simple rotations affect the implicit bias of Adam? Adela DePavia Vasileios Charisopoulos Rebecca Willett ODL 122 0 0 27 Oct 2025
Connectome-Guided Automatic Learning Rates for Deep Networks Peilin He Tananun Songdechakraiwut ODL OOD 143 0 0 27 Oct 2025
Towards Deep Physics-Informed Kolmogorov-Arnold Networks Spyros Rigas Fotios Anagnostopoulos M. Papachristou Georgios Alexandridis ODL AI4CE 140 0 0 27 Oct 2025
Randomness and Interpolation Improve Gradient DescentInternational Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2024 Jiawen Li Pascal Lefevre Anwar PP Abdul Majeed ODL 85 0 0 14 Oct 2025
Nonlinear discretizations and Newton's method: characterizing stationary points of regression objectives Conor Rowan ODL 100 0 0 13 Oct 2025
Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods Andrey Veprikov Arman Bolatov Samuel Horváth Aleksandr Beznosikov Martin Takáč Slavomír Hanzely ODL 148 0 0 12 Oct 2025
AutoGD: Automatic Learning Rate Selection for Gradient Descent Nikola Surjanovic Alexandre Bouchard-Côté Trevor Campbell ODL 63 0 0 10 Oct 2025
Convergence of optimizers implies eigenvalues filtering at equilibrium Jérôme Bolte Quoc-Tung Le Edouard Pauwels ODL 120 1 0 10 Oct 2025
Closed-Form Last Layer Optimization Alexandre Galashov Nathael Da Costa Liyuan Xu Philipp Hennig Arthur Gretton ODL 84 0 0 06 Oct 2025
Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization Kristi Topollai A. Choromańska ODL 136 0 0 06 Oct 2025
Explore the Loss space with Hill-ADAM Meenakshi Manikandan Leilani Gilpin ODL 89 0 0 04 Oct 2025
PENEX: AdaBoost-Inspired Neural Network Regularization Klaus-Rudolf Kladny Bernhard Schölkopf Michael Muehlebach ODL 164 0 0 02 Oct 2025
Stochastic Adaptive Gradient Descent Without Descent Jean-François Aujol Jérémie Bigot Camille Castera ODL 88 1 0 18 Sep 2025
Learning Neural Networks by Neuron Pursuit Akshay Kumar Jarvis Haupt ODL 63 0 0 15 Sep 2025
Gradient Methods with Online Scaling Part II. Practical Aspects Ya-Chi Chu Wenzhi Gao Yinyu Ye Madeleine Udell ODL 162 0 0 13 Sep 2025
Depth-Aware Initialization for Stable and Efficient Neural Network Training Vijay Pandey ODL 101 0 0 05 Sep 2025
Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks Hyungu Lee Taehyeong Kim Hayoung Choi ODL 99 0 0 30 Aug 2025
Flatness-aware Curriculum Learning via Adversarial Difficulty Hiroaki Aizawa Yoshikazu Hayashi ODL 80 0 0 26 Aug 2025
ANO : Faster is Better in Noisy Landscape Adrien Kegreisz ODL 184 0 0 25 Aug 2025
Enhanced NIRMAL Optimizer With Damped Nesterov Acceleration: A Comparative Analysis Nirmal Gaud Prasad Krishna Murthy Mostaque Md. Morshedur Hassan Abhijit Ganguly Vinay Mali Ms Lalita Bhagwat Randive Abhaypratap Singh ODL 91 0 0 22 Aug 2025
Fisher-Orthogonal Projection Methods for Natural Gradient Descent with Large Batches Yishun Lu Wesley Armour ODL 157 1 0 19 Aug 2025
Kourkoutas-Beta: A Sunspike-Driven Adam Optimizer with Desert Flair Stavros C. Kassinos ODL 137 0 0 18 Aug 2025
Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity Hikaru Umeda Hideaki Iiduka ODL 78 0 0 07 Aug 2025
ZetA: A Riemann Zeta-Scaled Extension of Adam for Deep Learning Samiksha BC ODL 72 0 0 01 Aug 2025
Dual Adaptivity: Universal Algorithms for Minimizing the Adaptive Regret of Convex Functions Lijun Zhang Wenhao Yang Guanghui Wang Wei Jiang Zhi Zhou ODL 49 0 0 01 Aug 2025
Principled Curriculum Learning using Parameter Continuation Methods Harsh Nilesh Pathak Randy Paffenroth ODL 101 1 0 29 Jul 2025
Dimer-Enhanced Optimization: A First-Order Approach to Escaping Saddle Points in Neural Network Training Yue Hu Zanxia Cao Yingchao Liu ODL 114 1 0 26 Jul 2025
Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization Serge Gratton Alena Kopaničáková Philippe Toint ODL 124 1 0 15 Jul 2025
Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding Qi Feng Yihong Liu Hinrich Schütze ODL 70 2 0 13 Jul 2025
An Adaptive Volatility-based Learning Rate Scheduler Kieran Chai Kai Ren ODL 67 0 0 11 Jul 2025
PDE-aware Optimizer for Physics-informed Neural Networks Hardik Shukla Manurag Khullar Vismay Churiwala ODL 66 0 0 10 Jul 2025
SoftSignSGD(S3): An Enhanced Optimizer for Practical DNN Training and Loss Spikes Minimization Beyond Adam Hanyang Peng Shuang Qin Yue Yu Fangqing Jiang Hui Wang Wen Gao ODL 73 1 0 09 Jul 2025
On the algorithmic construction of deep ReLU networks Daan Huybrechs ODL 144 0 0 23 Jun 2025
Hindsight-Guided Momentum (HGM) Optimizer: An Approach to Adaptive Learning Rate Krisanu Sarkar ODL 81 0 0 22 Jun 2025
PyLO: Towards Accessible Learned Optimizers in PyTorch Vaibhav Singh Benjamin Thérien Quentin G. Anthony Xiaolong Huang A. Moudgil Eugene Belilovsky ODL AI4CE 338 0 0 12 Jun 2025
MAC: An Efficient Gradient Preconditioning using Mean Activation Approximated Curvature Hyunseok Seung Jaewoo Lee Hyunsuk Ko ODL 121 1 0 10 Jun 2025
NysAct: A Scalable Preconditioned Gradient Descent using Nystrom ApproximationBigData Congress [Services Society] (BSS), 2024 Hyunseok Seung Jaewoo Lee Hyunsuk Ko ODL 133 0 0 10 Jun 2025
An Adaptive Method Stabilizing Activations for Enhanced Generalization Hyunseok Seung Jaewoo Lee Hyunsuk Ko ODL 146 0 0 10 Jun 2025
On the Stability of the Jacobian Matrix in Deep Neural Networks Benjamin Dadoun Soufiane Hayou Hanan Salam Abdalgader Abubaker Pierre Youssef ODL 134 0 0 10 Jun 2025
Adaptive Preconditioners Trigger Loss Spikes in Adam Zhiwei Bai Zhangchen Zhou Jiajie Zhao Xiaolong Li Zhiyu Li Feiyu Xiong Hongkang Yang Yaoyu Zhang Z. Xu ODL 192 0 0 05 Jun 2025
KOALA++: Efficient Kalman-Based Optimization with Gradient-Covariance Products Zixuan Xia A. Davtyan Paolo Favaro ODL 231 0 0 04 Jun 2025
PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning Arnulf Jentzen Julian Kranz Adrian Riekert ODL 179 0 0 28 May 2025
Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape Ioannis Bantzis James B. Simon Arthur Jacot ODL 176 1 0 27 May 2025
One-Time Soft Alignment Enables Resilient Learning without Weight Transport Jeonghwan Cheon Jaehyuk Bae Se-Bum Paik ODL 205 2 0 27 May 2025
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training Huishuai Zhang Bohan Wang Luoxin Chen ODL 336 0 0 22 May 2025
KO: Kinetics-inspired Neural Optimizer with PDE Simulation Approaches Mingquan Feng Yixin Huang Yifan Fu Shaobo Wang Junchi Yan ODL 89 0 0 20 May 2025
Self Distillation via Iterative Constructive Perturbations Maheak Dave Aniket K. Singh Aryan Pareek Harshita Jha Debasis Chaudhuri Manish P. Singh ODL 112 0 0 20 May 2025

Loading #Papers per Month with "ODL"

Past speakers

Name (-)

Top Contributors

Name (-)

Top Organizations at ResearchTrend.AI

Name (-)

Social Events

Date	Location	Event
No social events available