Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast
Convergence

Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence

24 February 2020

Simon Lacoste-Julien

Papers citing "Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence"

12 / 112 papers shown

Title
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap Lingxi Xie Xin Chen Kaifeng Bi Longhui Wei Yuhui Xu ... Lanfei Wang Anxiang Xiao Jianlong Chang Xiaopeng Zhang Qi Tian ViT 35 108 0 04 Aug 2020
Stochastic Hamiltonian Gradient Methods for Smooth Games Nicolas Loizou Hugo Berard Alexia Jolicoeur-Martineau Pascal Vincent Simon Lacoste-Julien Ioannis Mitliagkas 28 50 0 08 Jul 2020
A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images I. Laradji Pau Rodríguez López Oscar Manas Keegan Lensink M. Law Lironne Kurzman William Parker David Vazquez Derek Nowrouzezahrai 15 84 0 04 Jul 2020
LOOC: Localize Overlapping Objects with Count Supervision I. Laradji Rafael Pardiñas Pau Rodríguez López David Vazquez 17 10 0 03 Jul 2020
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization Ahmed Khaled Othmane Sebbouh Nicolas Loizou Robert Mansel Gower Peter Richtárik 11 45 0 20 Jun 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 25 74 0 18 Jun 2020
Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search) Sharan Vaswani I. Laradji Frederik Kunstner S. Meng Mark W. Schmidt Simon Lacoste-Julien 19 27 0 11 Jun 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 39 491 0 23 Mar 2020
Training Neural Networks for and by Interpolation Leonard Berrada Andrew Zisserman M. P. Kumar 3DH 8 60 0 13 Jun 2019
L4: Practical loss-based stepsize adaptation for deep learning Michal Rolínek Georg Martius ODL 36 63 0 14 Feb 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 136 1,198 0 16 Aug 2016
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 99 570 0 08 Dec 2012