Papers citing 'Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent'

Title
Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize Tong Zhang Chris Junchi Li 256 0 0 15 Jul 2024
An optimal scheduled learning rate for a randomized Kaczmarz algorithmSIAM Journal on Matrix Analysis and Applications (SIMAX), 2022 Nicholas F. Marshall Oscar Mickelin 272 11 0 24 Feb 2022
Optimal learning rate schedules in high-dimensional non-convex optimization problems Stéphane dÁscoli Maria Refinetti Giulio Biroli 150 9 0 09 Feb 2022
On Generalization of Adaptive Methods for Over-parameterized Linear Regression Vatsal Shah Soumya Basu Anastasios Kyrillidis Sujay Sanghavi AI4CE 101 4 0 28 Nov 2020
A Survey on Large-scale Machine LearningIEEE Transactions on Knowledge and Data Engineering (TKDE), 2020 Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 166 139 0 10 Aug 2020
Weak error analysis for stochastic gradient descent optimization algorithms A. Bercher Lukas Gonon Arnulf Jentzen Diyora Salimova 222 4 0 03 Jul 2020
Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation Beidi Chen Yingchen Xu Anshumali Shrivastava 148 16 0 30 Oct 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective Guan-Horng Liu Evangelos A. Theodorou AI4CE 250 74 0 28 Aug 2019
Optimizing Multi-GPU Parallelization Strategies for Deep Learning TrainingIEEE Micro (IEEE Micro), 2019 Saptadeep Pal Eiman Ebrahimi A. Zulfiqar Yaosheng Fu Victor Zhang Szymon Migacz D. Nellans Puneet Gupta 235 66 0 30 Jul 2019
Convergence rates for the stochastic gradient descent method for non-convex objective functions Benjamin J. Fehrman Benjamin Gess Arnulf Jentzen 282 109 0 02 Apr 2019
CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers A. Koliousis Pijika Watcharapichat Matthias Weidlich Kai Zou Paolo Costa Peter R. Pietzuch 166 70 0 08 Jan 2019
Minimum weight norm models do not always generalize well for over-parameterized problems Vatsal Shah Anastasios Kyrillidis Sujay Sanghavi 220 21 0 16 Nov 2018
Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent Matteo Pirotta Marcello Restelli 68 1 0 09 Dec 2017
Convergence diagnostics for stochastic gradient descent with constant step size Jerry Chee Panos Toulis 171 12 0 17 Oct 2017
Scalable Nonlinear AUC Maximization Methods Majdi Khalid I. Ray H. Chitsaz 142 0 0 02 Oct 2017
Analysis and Optimization of Convolutional Neural Network Architectures Martin Thoma 157 75 0 31 Jul 2017
Insensitive Stochastic Gradient Twin Support Vector Machine for Large Scale Problems Zhen Wang Yifei Shao Lan Bai Li-Ming Liu N. Deng 113 40 0 19 Apr 2017
Deep LSTM for Large Vocabulary Continuous Speech Recognition Xudong Tian Jun Zhang Zejun Ma Yi He Juan Wei Peihao Wu Wenchang Situ Shuai Li Yang Zhang 95 30 0 21 Mar 2017
Exponential Moving Average Model in Parallel Speech Recognition Training Xudong Tian Jun Zhang Zejun Ma Yi He Juan Wei 90 4 0 03 Mar 2017
e-Distance Weighted Support Vector Regression Yan Wang G. Ou Wei Pang Lan Huang George Macleod Coghill 158 1 0 21 Jul 2016
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 721 3,492 0 15 Jun 2016
On the Convergence of A Family of Robust Losses for Stochastic Gradient Descent Bo Han Ivor W. Tsang Ling-Hao Chen NoLa 385 22 0 05 May 2016
Optimal Margin Distribution Machine Teng Zhang Zhi Zhou 141 76 0 12 Apr 2016
Stochastic modified equations and adaptive stochastic gradient algorithms Qianxiao Li Cheng Tai E. Weinan 258 307 0 19 Nov 2015
Stochastic gradient descent methods for estimation with large data sets Dustin Tran Panos Toulis E. Airoldi 136 14 0 22 Sep 2015
A Bayesian Approach for Online Classifier Ensemble Qinxun Bai Henry Lam Stan Sclaroff BDL 100 0 0 08 Jul 2015
Towards stability and optimality in stochastic gradient descent Panos Toulis Dustin Tran E. Airoldi 256 58 0 10 May 2015
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields Mark Schmidt Reza Babanezhad Mohamed Osama Ahmed Aaron Defazio Ann Clifton Anoop Sarkar 159 83 0 16 Apr 2015
Asymptotic and finite-sample properties of estimators based on stochastic gradients Panos Toulis E. Airoldi 215 12 0 13 Aug 2014
Unit Tests for Stochastic OptimizationInternational Conference on Learning Representations (ICLR), 2013 Tom Schaul Ioannis Antonoglou David Silver 233 92 0 20 Dec 2013
An Empirical Evaluation of Sequence-Tagging Trainers B. Palaniappan S. Shevade S. Sundararajan S. Keerthi 80 2 0 11 Nov 2013
Large Margin Distribution MachineKnowledge Discovery and Data Mining (KDD), 2013 Teng Zhang Zhi Zhou 181 112 0 05 Nov 2013
No More Pesky Learning RatesInternational Conference on Machine Learning (ICML), 2012 Tom Schaul Sixin Zhang Yann LeCun 446 489 0 06 Jun 2012
Stochastic Smoothing for Nonsmooth Minimizations: Accelerating SGD by Exploiting StructureInternational Conference on Machine Learning (ICML), 2012 H. Ouyang Alexander G. Gray 256 28 0 21 May 2012
b-Bit Minwise Hashing in Practice: Large-Scale Batch and Online Learning and Using GPUs for Fast Preprocessing with Simple Hash Functions Ping Li Anshumali Shrivastava A. König 103 6 0 14 May 2012
Learning Feature Hierarchies with Centered Deep Boltzmann Machines G. Montavon K. Müller BDL AI4CE 111 9 0 16 Mar 2012