Barzilai-Borwein Step Size for Stochastic Gradient Descent

13 May 2016

Papers citing "Barzilai-Borwein Step Size for Stochastic Gradient Descent"

25 / 25 papers shown

Title
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 55 4 0 06 Jun 2024
Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning Achraf Bahamou D. Goldfarb ODL 36 0 0 23 May 2023
Distributed and Scalable Optimization for Robust Proton Treatment Planning A. Fu V. Taasti M. Zarepisheh 24 2 0 27 Apr 2023
Stochastic Steffensen method Minda Zhao Zehua Lai Lek-Heng Lim ODL 15 3 0 28 Nov 2022
Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization Ali Kavis Stratis Skoulakis Kimon Antonakopoulos L. Dadi V. Cevher 32 15 0 03 Nov 2022
Adaptive scaling of the learning rate by second order automatic differentiation F. Gournay Alban Gossard ODL 31 1 0 26 Oct 2022
A Stochastic Variance Reduced Gradient using Barzilai-Borwein Techniques as Second Order Information Hardik Tankaria N. Yamashita 23 1 0 23 Aug 2022
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms Binghui Xie Chen Jin Kaiwen Zhou James Cheng Wei Meng 45 1 0 28 Apr 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 26 4 0 29 Jan 2022
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization Kaiwen Zhou Anthony Man-Cho So James Cheng 29 1 0 30 Sep 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark Schmidt Simon Lacoste-Julien 23 18 0 18 Feb 2021
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 47 162 0 03 Jul 2020
Balancing Rates and Variance via Adaptive Batch-Size for Stochastic Optimization Problems Zhan Gao Alec Koppel Alejandro Ribeiro 33 10 0 02 Jul 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 29 181 0 24 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 30 168 0 19 Dec 2019
Fast Stochastic Ordinal Embedding with Variance Reduction and Adaptive Step Size Ke Ma Jinshan Zeng Qianqian Xu Xiaochun Cao Wei Liu Yuan Yao 36 3 0 01 Dec 2019
Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG Ran Xin U. Khan S. Kar 22 8 0 08 Oct 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Simon Lacoste-Julien ODL 50 205 0 24 May 2019
Block stochastic gradient descent for large-scale tomographic reconstruction in a parallel network Yushan Gao A. Biguri T. Blumensath 34 3 0 28 Mar 2019
Dual optimization for convex constrained objectives without the gradient-Lipschitz assumption Martin Bompaire Emmanuel Bacry Stéphane Gaïffas 30 6 0 10 Jul 2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes Rachel A. Ward Xiaoxia Wu Léon Bottou ODL 27 361 0 05 Jun 2018
SPSA-FSR: Simultaneous Perturbation Stochastic Approximation for Feature Selection and Ranking Zeren D. Yenice Niranjan Adhikari Yong Kai Wong V. Aksakalli A. T. Gumus B. Abbasi 27 8 0 16 Apr 2018
Block-Cyclic Stochastic Coordinate Descent for Deep Neural Networks Kensuke Nakamura Stefano Soatto Byung-Woo Hong BDL ODL 43 6 0 20 Nov 2017
Big Batch SGD: Automated Inference using Adaptive Batch Sizes Soham De A. Yadav David Jacobs Tom Goldstein ODL 37 62 0 18 Oct 2016
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 93 737 0 19 Mar 2014