v1v2 (latest)

On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants

23 June 2015

Papers citing "On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants"

50 / 101 papers shown

Title
An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning Cen-Jhih Li Aditya Bhaskara 358 0 0 17 Feb 2025
Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction Yury Demidovich Grigory Malinovsky Peter Richtárik 213 3 0 11 Mar 2024
Federated Empirical Risk Minimization via Second-Order Method S. Bian Zhao Song Junze Yin FedML 220 10 0 27 May 2023
Learning Large Scale Sparse Models A. Dhingra Jie Shen Nicholas Kleene 143 0 0 26 Jan 2023
Scaling up Stochastic Gradient Descent for Non-convex OptimisationMachine-mediated learning (ML), 2022 S. Mohamad H. Alamri A. Bouchachia 172 3 0 06 Oct 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing ClustersACM Interational Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), 2022 Zhuqing Liu Xin Zhang Jia-Wei Liu 201 1 0 17 Aug 2022
Adaptive Stochastic Gradient Descent for Fast and Communication-Efficient Distributed Learning Serge Kas Hanna Rawad Bitar Parimal Parag Venkateswara Dasari S. E. Rouayheb 207 4 0 04 Aug 2022
Adaptive Sketches for Robust Regression with Importance SamplingInternational Workshop and International Workshop on Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM), 2022 S. Mahabadi David P. Woodruff Samson Zhou 117 6 0 16 Jul 2022
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 273 9 0 05 May 2022
Distributed Dynamic Safe Screening Algorithms for Sparse Regularization Runxue Bao Xidong Wu Wenhan Xian Heng-Chiao Huang 162 1 0 23 Apr 2022
Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic ProgrammingMathematical programming (Math. Program.), 2021 Sen Na M. Anitescu Mladen Kolar 225 42 0 23 Sep 2021
Asynchronous Stochastic Optimization Robust to Arbitrary DelaysNeural Information Processing Systems (NeurIPS), 2021 Alon Cohen Amit Daniely Yoel Drori Tomer Koren Mariano Schain 212 39 0 22 Jun 2021
Memory Augmented Optimizers for Deep LearningInternational Conference on Learning Representations (ICLR), 2021 Paul-Aymeric McRae Prasanna Parthasarathi Mahmoud Assran Sarath Chandar ODL 122 5 0 20 Jun 2021
Federated Learning with Buffered Asynchronous AggregationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2021 John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 319 389 0 11 Jun 2021
LocalNewton: Reducing Communication Bottleneck for Distributed Learning Vipul Gupta Avishek Ghosh Michal Derezinski Rajiv Khanna Kannan Ramchandran Michael W. Mahoney 168 13 0 16 May 2021
Asynchronous Parallel Stochastic Quasi-Newton MethodsParallel Computing (PC), 2020 Qianqian Tong Guannan Liang Xingyu Cai Chunjiang Zhu J. Bi ODL 182 10 0 02 Nov 2020
Asynchronous Distributed Optimization with Stochastic Delays Margalit Glasgow Mary Wootters 220 3 0 22 Sep 2020
Beyond variance reduction: Understanding the true impact of baselines on policy optimizationInternational Conference on Machine Learning (ICML), 2020 Wesley Chung Valentin Thomas Marlos C. Machado Nicolas Le Roux OffRL 376 31 0 31 Aug 2020
Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentum Jerry Chee Ping Li 124 13 0 27 Aug 2020
Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization Chaobing Song Yong Jiang Yi-An Ma 437 24 0 18 Jun 2020
Convergence of Recursive Stochastic Algorithms using Wasserstein DivergenceSIAM Journal on Mathematics of Data Science (SIMODS), 2020 Abhishek Gupta W. Haskell 128 5 0 25 Mar 2020
Adaptive Distributed Stochastic Gradient Descent for Minimizing Delay in the Presence of StragglersIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020 Serge Kas Hanna Rawad Bitar Parimal Parag Venkateswara Dasari S. E. Rouayheb 155 16 0 25 Feb 2020
Adaptive Sampling Distributed Stochastic Variance Reduced Gradient for Heterogeneous Distributed Datasets Ilqar Ramazanli Han Nguyen Hai Pham Sashank J. Reddi Barnabás Póczós 196 11 0 20 Feb 2020
Faster On-Device Training Using New Federated Momentum Algorithm Zhouyuan Huo Qian Yang Bin Gu Heng-Chiao Huang FedML 276 52 0 06 Feb 2020
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine AttacksIEEE Transactions on Signal Processing (IEEE Trans. Signal Process.), 2019 Zhaoxian Wu Qing Ling Tianyi Chen G. Giannakis FedML AAML 209 216 0 29 Dec 2019
Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization Fanhua Shang Bingkun Wei Hongying Liu Yuanyuan Liu Jiacheng Zhuo 132 1 0 02 Dec 2019
Aggregated Gradient Langevin DynamicsAAAI Conference on Artificial Intelligence (AAAI), 2019 Chao Zhang Jiahao Xie Zebang Shen P. Zhao Tengfei Zhou Hui Qian 189 1 0 21 Oct 2019
Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm BallsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2019 Jiacheng Zhuo Qi Lei A. Dimakis Constantine Caramanis 60 4 0 17 Oct 2019
Automatic and Simultaneous Adjustment of Learning Rate and Momentum for Stochastic Gradient DescentIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019 Tomer Lancewicki Selçuk Köprü 83 5 0 20 Aug 2019
Calibrating the Adaptive Learning Rate to Improve Convergence of ADAM Qianqian Tong Guannan Liang J. Bi 214 7 0 02 Aug 2019
DEAM: Adaptive Momentum with Discriminative Weight for Stochastic OptimizationInternational Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2019 Jiyang Bai Yuxiang Ren Jiawei Zhang ODL 183 1 0 25 Jul 2019
ASYNC: A Cloud Engine with Asynchrony and History for Distributed Machine LearningIEEE International Parallel and Distributed Processing Symposium (IPDPS), 2019 Saeed Soori Bugra Can Mert Gurbuzbalaban M. Dehnavi GNN OffRL 262 4 0 19 Jul 2019
A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operators Xun Zhang W. Haskell Z. Ye 160 3 0 22 Jun 2019
Scaling Up Quasi-Newton Algorithms: Communication Efficient Distributed SR1International Conference on Machine Learning, Optimization, and Data Science (MOD), 2019 Majid Jahani M. Nazari S. Rusakov A. Berahas Martin Takávc 202 16 0 30 May 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 138 23 0 26 Apr 2019
Communication trade-offs for synchronized distributed SGD with large step size Kumar Kshitij Patel Hadrien Hendrikx FedML 162 27 0 25 Apr 2019
OverSketched Newton: Fast Convex Optimization for Serverless Systems Vipul Gupta S. Kadhe T. Courtade Michael W. Mahoney Kannan Ramchandran 200 34 0 21 Mar 2019
Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Training Chengjie Li Ruixuan Li Yining Qi Yuhua Li Pan Zhou Song Guo Keqin Li 239 17 0 21 Feb 2019
Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems Ehsan Kazemi Liqiang Wang 101 2 0 05 Feb 2019
Uniform-in-Time Weak Error Analysis for Stochastic Gradient Descent Algorithms via Diffusion ApproximationCommunications in Mathematical Sciences (Comm. Math. Sci.), 2019 Yuanyuan Feng Tingran Gao Lei Li Jian‐Guo Liu Yulong Lu 179 25 0 02 Feb 2019
Fundamental Limits of Approximate Gradient Coding Sinong Wang Jiashang Liu Ness B. Shroff 175 44 0 23 Jan 2019
Stochastic Doubly Robust Gradient Kanghoon Lee Jihye Choi Moonsu Cha Jung Kwon Lee Tae-Yoon Kim 80 0 0 21 Dec 2018
Asynchronous Stochastic Composition Optimization with Variance Reduction Shuheng Shen Linli Xu Jingchang Liu Junliang Guo Qing Ling 116 2 0 15 Nov 2018
ASVRG: Accelerated Proximal SVRG Fanhua Shang L. Jiao Kaiwen Zhou James Cheng Yan Ren Yufei Jin ODL 173 33 0 07 Oct 2018
Fast Variance Reduction Method with Stochastic Batch Size Xuanqing Liu Cho-Jui Hsieh 197 6 0 07 Aug 2018
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence RatesInternational Conference on Machine Learning (ICML), 2018 Kaiwen Zhou Fanhua Shang James Cheng 170 78 0 28 Jun 2018
Federated Learning with Non-IID Data Yue Zhao Meng Li Liangzhen Lai Naveen Suda Damon Civin Vikas Chandra FedML 500 2,946 0 02 Jun 2018
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 330 59 0 25 May 2018
Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products Zhe-Cheng Fan T. Chan Yi-Hsuan Yang J. Jang 145 7 0 24 May 2018
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning Xin Zhang Jia-Wei Liu Zhengyuan Zhu 157 17 0 24 May 2018