Stop Wasting My Gradients: Practical SVRG

5 November 2015

Mohamed Osama Ahmed

Papers citing "Stop Wasting My Gradients: Practical SVRG"

38 / 38 papers shown

Title
An Effective Dynamic Gradient Calibration Method for Continual Learning Weichen Lin Jiaxiang Chen Ru Huang Huihua Ding CLL 46 0 0 30 Jul 2024
SOREL: A Stochastic Algorithm for Spectral Risks Minimization Yuze Ge Rujun Jiang 47 0 0 19 Jul 2024
Stochastic Ratios Tracking Algorithm for Large Scale Machine Learning Problems Shigeng Sun Yuchen Xie 18 3 0 17 May 2023
Stochastic Steffensen method Minda Zhao Zehua Lai Lek-Heng Lim ODL 15 3 0 28 Nov 2022
Variance Reduced Training with Stratified Sampling for Forecasting Models Yucheng Lu Youngsuk Park Lifan Chen Bernie Wang Christopher De Sa Dean Phillips Foster AI4TS 38 17 0 02 Mar 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark Schmidt Simon Lacoste-Julien 23 18 0 18 Feb 2021
Variance-Reduced Methods for Machine Learning Robert Mansel Gower Mark Schmidt Francis R. Bach Peter Richtárik 24 112 0 02 Oct 2020
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm Majid Jahani M. Nazari R. Tappenden A. Berahas Martin Takávc ODL 19 10 0 06 Jun 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 29 181 0 24 Feb 2020
Sampling and Update Frequencies in Proximal Variance-Reduced Stochastic Gradient Methods Martin Morin Pontus Giselsson 27 4 0 13 Feb 2020
History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms Kaiyi Ji Zhe Wang Bowen Weng Yi Zhou Wei Zhang Yingbin Liang ODL 18 5 0 21 Oct 2019
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction Pan Xu F. Gao Quanquan Gu 33 83 0 18 Sep 2019
Reducing the variance in online optimization by transporting past gradients Sébastien M. R. Arnold Pierre-Antoine Manzagol Reza Babanezhad Ioannis Mitliagkas Nicolas Le Roux 26 28 0 08 Jun 2019
An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradient Pan Xu F. Gao Quanquan Gu 21 93 0 29 May 2019
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization Nhan H. Pham Lam M. Nguyen Dzung Phan Quoc Tran-Dinh 18 139 0 15 Feb 2019
Continuous-time Models for Stochastic Optimization Algorithms Antonio Orvieto Aurelien Lucchi 19 31 0 05 Oct 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Stochastic Variance-Reduced Policy Gradient Matteo Papini Damiano Binaghi Giuseppe Canonaco Matteo Pirotta Marcello Restelli 21 174 0 14 Jun 2018
Doubly Robust Bayesian Inference for Non-Stationary Streaming Data with $β$ -Divergences Jeremias Knoblauch Jack Jewson Theodoros Damoulas 25 56 0 06 Jun 2018
Stochastic Zeroth-order Optimization via Variance Reduction method L. Liu Minhao Cheng Cho-Jui Hsieh Dacheng Tao 24 19 0 30 May 2018
Distributed Stochastic Optimization via Adaptive SGD Ashok Cutkosky R. Busa-Fekete FedML 35 21 0 16 Feb 2018
AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks Aditya Devarakonda Maxim Naumov M. Garland ODL 24 136 0 06 Dec 2017
Variance-Reduced Stochastic Learning under Random Reshuffling Bicheng Ying Kun Yuan Ali H. Sayed 33 13 0 04 Aug 2017
Coupling Adaptive Batch Sizes with Learning Rates Lukas Balles Javier Romero Philipp Hennig ODL 21 110 0 15 Dec 2016
Optimization for Large-Scale Machine Learning with Distributed Features and Observations A. Nathan Diego Klabjan 32 13 0 31 Oct 2016
Big Batch SGD: Automated Inference using Adaptive Batch Sizes Soham De A. Yadav David Jacobs Tom Goldstein ODL 37 62 0 18 Oct 2016
Federated Optimization: Distributed Machine Learning for On-Device Intelligence Jakub Konecný H. B. McMahan Daniel Ramage Peter Richtárik FedML 71 1,877 0 08 Oct 2016
Understanding intermediate layers using linear classifier probes Guillaume Alain Yoshua Bengio FAtt 53 900 0 05 Oct 2016
Exact and Inexact Subsampled Newton Methods for Optimization Raghu Bollapragada R. Byrd J. Nocedal 23 178 0 27 Sep 2016
Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method Lihua Lei Michael I. Jordan 29 96 0 12 Sep 2016
Stochastic Variance Reduction Methods for Saddle-Point Problems B. Palaniappan Francis R. Bach 23 210 0 20 May 2016
Barzilai-Borwein Step Size for Stochastic Gradient Descent Conghui Tan Shiqian Ma Yuhong Dai Yuqiu Qian 45 182 0 13 May 2016
Trading-off variance and complexity in stochastic gradient descent Vatsal Shah Megasthenis Asteris Anastasios Kyrillidis Sujay Sanghavi 27 13 0 22 Mar 2016
Importance Sampling for Minibatches Dominik Csiba Peter Richtárik 32 113 0 06 Feb 2016
SCOPE: Scalable Composite Optimization for Learning on Spark Shen-Yi Zhao Ru Xiang Yinghuan Shi Peng Gao Wu-Jun Li 32 16 0 30 Jan 2016
Online Batch Selection for Faster Training of Neural Networks I. Loshchilov Frank Hutter ODL 53 298 0 19 Nov 2015
Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectives Zeyuan Allen-Zhu Yang Yuan 37 195 0 05 Jun 2015
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 93 737 0 19 Mar 2014