A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models

16 May 2025

Papers citing "A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models"

24 / 24 papers shown

Title
Gradient descent with adaptive stepsize converges (nearly) linearly under fourth-order growth Damek Davis Dmitriy Drusvyatskiy L. Jiang 21 2 0 29 Sep 2024
Gradient Descent Provably Solves Nonlinear Tomographic Reconstruction Sara Fridovich-Keil Fabrizio Valdivia Gordon Wetzstein Benjamin Recht Mahdi Soltanolkotabi 28 4 0 06 Oct 2023
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 57 13 0 22 Apr 2022
Convergence of gradient descent for learning linear neural networks Gabin Maxime Nguegnang Holger Rauhut Ulrich Terstiege MLT 30 17 0 04 Aug 2021
Overparameterization of deep ResNet: zero loss and mean-field analysis Zhiyan Ding Shi Chen Qin Li S. Wright ODL 65 25 0 30 May 2021
Convergence and Implicit Bias of Gradient Flow on Overparametrized Linear Networks Hancheng Min Salma Tarmoun René Vidal Enrique Mallada MLT 21 5 0 13 May 2021
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Chaoyue Liu Libin Zhu M. Belkin ODL 41 258 0 29 Feb 2020
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology Quynh N. Nguyen Marco Mondelli ODL AI4CE 31 67 0 18 Feb 2020
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks Gauthier Gidel Francis R. Bach Simon Lacoste-Julien AI4CE 51 153 0 30 Apr 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 146 1,089 0 18 Feb 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 55 94 0 24 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 87 823 0 19 Dec 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 100 284 0 04 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 151 1,261 0 04 Oct 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 188 3,160 0 20 Jun 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 119 239 0 04 Jun 2018
On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport Lénaïc Chizat Francis R. Bach OT 157 731 0 24 May 2018
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Andrea Montanari Phan-Minh Nguyen MLT 76 855 0 18 Apr 2018
Tensor2Tensor for Neural Machine Translation Ashish Vaswani Samy Bengio E. Brevdo François Chollet Aidan Gomez ... Nal Kalchbrenner Niki Parmar Ryan Sepassi Noam M. Shazeer Jakob Uszkoreit 81 528 0 16 Mar 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark Schmidt 221 1,208 0 16 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 200 18,534 0 06 Feb 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 128 1,830 0 20 Dec 2013
Spectral Compressed Sensing via Structured Matrix Completion Yuxin Chen Yuejie Chi 44 58 0 16 Apr 2013