Probabilistic Line Searches for Stochastic Optimization

10 February 2015

Papers citing "Probabilistic Line Searches for Stochastic Optimization"

32 / 32 papers shown

Title
Stochastic Subspace Descent Accelerated via Bi-fidelity Line Search Nuojin Cheng Alireza Doostan Stephen Becker 46 0 0 30 Apr 2025
Faster Convergence for Transformer Fine-tuning with Line Search Methods Philip Kenneweg Leonardo Galli Tristan Kenneweg Barbara Hammer ODL 51 2 0 27 Mar 2024
Learning-Rate-Free Learning: Dissecting D-Adaptation and Probabilistic Line Search Max McGuinness ODL 22 0 0 06 Aug 2023
Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning Achraf Bahamou D. Goldfarb ODL 36 0 0 23 May 2023
Local Bayesian optimization via maximizing probability of descent Quan Nguyen Kaiwen Wu Jacob R. Gardner Roman Garnett 25 23 0 21 Oct 2022
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 58 2 0 07 Jul 2021
Large Scale Private Learning via Low-rank Reparametrization Da Yu Huishuai Zhang Wei Chen Jian Yin Tie-Yan Liu 29 101 0 17 Jun 2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Yuchen Jin Dinesh Manocha Liangyu Zhao Yibo Zhu Chuanxiong Guo Marco Canini Arvind Krishnamurthy 37 18 0 22 May 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 46 6 0 09 Nov 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 38 0 0 02 Oct 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 47 162 0 03 Jul 2020
Adaptive Stochastic Optimization Frank E. Curtis K. Scheinberg ODL 19 29 0 18 Jan 2020
Empirical study towards understanding line search approximations for training neural networks Younghwan Chae D. Wilke 27 11 0 15 Sep 2019
Stochastic quasi-Newton with line-search regularization A. Wills Thomas B. Schon ODL 29 21 0 03 Sep 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Simon Lacoste-Julien ODL 41 205 0 24 May 2019
DeepOBS: A Deep Learning Optimizer Benchmark Suite Frank Schneider Lukas Balles Philipp Hennig ODL 38 71 0 13 Mar 2019
LOSSGRAD: automatic learning rate in gradient descent B. Wójcik Lukasz Maziarka Jacek Tabor ODL 42 4 0 20 Feb 2019
A Modern Retrospective on Probabilistic Numerics Chris J. Oates T. Sullivan AI4CE 24 64 0 14 Jan 2019
A fast quasi-Newton-type method for large-scale stochastic optimisation A. Wills Carl Jidling Thomas B. Schon ODL 36 7 0 29 Sep 2018
Particle Filtering Methods for Stochastic Optimization with Application to Large-Scale Empirical Risk Minimization Bin Liu 30 10 0 23 Jul 2018
The Incremental Proximal Method: A Probabilistic Perspective Ömer Deniz Akyildiz Victor Elvira Joaquín Míguez 11 7 0 12 Jul 2018
Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models Hugh Salimbeni Stefanos Eleftheriadis J. Hensman BDL 30 85 0 24 Mar 2018
Learning nonlinear state-space models using smooth particle-filter-based likelihood approximations Andreas Svensson Fredrik Lindsten Thomas B. Schon 11 6 0 29 Nov 2017
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients Lukas Balles Philipp Hennig 31 161 0 22 May 2017
Early Stopping without a Validation Set Maren Mahsereci Lukas Balles Christoph Lassner Philipp Hennig 26 87 0 28 Mar 2017
Bayesian Probabilistic Numerical Methods Jon Cockayne Chris J. Oates T. Sullivan Mark Girolami 27 164 0 13 Feb 2017
Coupling Adaptive Batch Sizes with Learning Rates Lukas Balles Javier Romero Philipp Hennig ODL 21 110 0 15 Dec 2016
An empirical analysis of the optimization of deep network loss surfaces Daniel Jiwoong Im Michael Tao K. Branson ODL 35 62 0 13 Dec 2016
Big Batch SGD: Automated Inference using Adaptive Batch Sizes Soham De A. Yadav David Jacobs Tom Goldstein ODL 37 62 0 18 Oct 2016
A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation T. Bui Josiah Yan Richard Turner 26 25 0 23 May 2016
Barzilai-Borwein Step Size for Stochastic Gradient Descent Conghui Tan Shiqian Ma Yuhong Dai Yuqiu Qian 40 182 0 13 May 2016
Automatic Differentiation Variational Inference A. Kucukelbir Dustin Tran Rajesh Ranganath Andrew Gelman David M. Blei 47 710 0 02 Mar 2016