v1v2 (latest)

Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains

20 July 2017

Papers citing "Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains"

50 / 106 papers shown

Title
Sharp Gaussian approximations for Decentralized Federated Learning Soham Bonnerjee Sayar Karmakar Wei Biao Wu FedML 83 0 0 12 May 2025
Towards Weaker Variance Assumptions for Stochastic Optimization Ahmet Alacaoglu Yura Malitsky Stephen J. Wright 82 1 0 14 Apr 2025
A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression Yixuan Zhang Dongyan Yudong Chen Qiaomin Xie 58 0 0 11 Apr 2025
Randomised Splitting Methods and Stochastic Gradient Descent Luke Shaw Peter A. Whalley 104 1 0 05 Apr 2025
Online Inference for Quantiles by Constant Learning-Rate Stochastic Gradient Descent Ziyang Wei Jiaqi Li Likai Chen Wei Biao Wu 114 0 0 04 Mar 2025
Reinforcement Teaching Alex Lewandowski Calarina Muslimani Dale Schuurmans Matthew E. Taylor Jun Luo 194 2 0 28 Jan 2025
Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent Xiang Li Qiaomin Xie 105 1 0 15 Dec 2024
Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way Jeongyeol Kwon Luke Dotson Yudong Chen Qiaomin Xie 79 1 0 16 Oct 2024
Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson-Romberg Extrapolation Marina Sheshukova Denis Belomestny Alain Durmus Eric Moulines Alexey Naumov S. Samsonov 75 1 0 07 Oct 2024
Can LLMs predict the convergence of Stochastic Gradient Descent? Hiroki Sakaji Abdelhakim Benechehab Wataru Kuramoto LRM 95 2 0 03 Aug 2024
Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize Tong Zhang Chris Junchi Li 76 0 0 15 Jul 2024
Effect of Random Learning Rate: Theoretical Analysis of SGD Dynamics in Non-Convex Optimization via Stationary Distribution Naoki Yoshida Shogo H. Nakakita Masaaki Imaizumi 62 1 0 23 Jun 2024
An Analysis of Elo Rating Systems via Markov Chains Sam Olesker-Taylor Luca Zanetti 114 1 0 09 Jun 2024
Computing the Bias of Constant-step Stochastic Approximation with Markovian Noise Sebastian Allmeier Nicolas Gast 85 5 0 23 May 2024
Score-based Generative Models with Adaptive Momentum Ziqing Wen Xiaoge Deng Ping Luo Tao Sun Dongsheng Li DiffM 57 0 0 22 May 2024
Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA Yixuan Zhang D. Huo Yudong Chen Qiaomin Xie 77 2 0 09 Apr 2024
A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma ... Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang 68 9 0 17 Mar 2024
Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning R. Srikant 115 6 0 28 Jan 2024
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation Yixuan Zhang Qiaomin Xie 80 6 0 25 Jan 2024
Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference D. Huo Yudong Chen Qiaomin Xie 64 4 0 18 Dec 2023
Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent Krunoslav Lehman Pavasovic Alain Durmus Umut Simsekli OffRL 33 2 0 27 Oct 2023
Robust Stochastic Optimization via Gradient Quantile Clipping Ibrahim Merad Stéphane Gaïffas 79 2 0 29 Sep 2023
Online covariance estimation for stochastic gradient descent under Markovian sampling Abhishek Roy Krishnakumar Balasubramanian 97 6 0 03 Aug 2023
Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing? Kyurae Kim Yian Ma Jacob R. Gardner 103 7 0 27 Jul 2023
Stochastic Methods in Variational Inequalities: Ergodicity, Bias and Refinements Emmanouil-Vasileios Vlatakis-Gkaragkounis Angeliki Giannou Yudong Chen Qiaomin Xie 52 4 0 28 Jun 2023
Convergence and concentration properties of constant step-size SGD through Markov chains Ibrahim Merad Stéphane Gaïffas 68 5 0 20 Jun 2023
Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often! Romain Chor Abdellatif Zaidi Milad Sefidgaran Yijun Wan FedML 76 7 0 09 Jun 2023
Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGD Luca Arnaboldi Florent Krzakala Bruno Loureiro Ludovic Stephan MLT 99 5 0 29 May 2023
Statistical Analysis of Fixed Mini-Batch Gradient Descent Estimator Haobo Qi Feifei Wang Hansheng Wang 61 13 0 13 Apr 2023
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance Krishnakumar Balasubramanian Promit Ghosal Ye He 107 5 0 03 Apr 2023
On the Utility of Equal Batch Sizes for Inference in Stochastic Gradient Descent Rahul Singh A. Shukla Dootika Vats 56 0 0 14 Mar 2023
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression Bhavya Agrawalla Krishnakumar Balasubramanian Promit Ghosal 107 2 0 20 Feb 2023
Why is parameter averaging beneficial in SGD? An objective smoothing perspective Atsushi Nitanda Ryuhei Kikuchi Shugo Maeda Denny Wu FedML 49 0 0 18 Feb 2023
From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks Luca Arnaboldi Ludovic Stephan Florent Krzakala Bruno Loureiro MLT 105 34 0 12 Feb 2023
Iterative regularization in classification via hinge loss diagonal descent Vassilis Apidopoulos T. Poggio Lorenzo Rosasco S. Villa 63 2 0 24 Dec 2022
Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes D. Huo Yudong Chen Qiaomin Xie 85 17 0 03 Oct 2022
$Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers$ Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$ -Covers Sejun Park Umut Simsekli Murat A. Erdogdu 107 9 0 19 Sep 2022
Efficiency Ordering of Stochastic Gradient Descent Jie Hu Vishwaraj Doshi Do Young Eun 78 7 0 15 Sep 2022
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 91 11 0 16 Aug 2022
Tuning Stochastic Gradient Algorithms for Statistical Inference via Large-Sample Asymptotics Jeffrey Negrea Jun Yang Haoyue Feng Daniel M. Roy Jonathan H. Huggins 60 1 0 25 Jul 2022
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 135 59 0 08 Jun 2022
Estimation and Inference by Stochastic Optimization Jean-Jacques Forneron 79 5 0 06 May 2022
A Variance-Reduced Stochastic Accelerated Primal Dual Algorithm Bugra Can Mert Gurbuzbalaban N. Aybat 52 5 0 19 Feb 2022
Online Learning to Transport via the Minimal Selection Principle Wenxuan Guo Y. Hur Tengyuan Liang Christopher Ryan 53 3 0 09 Feb 2022
Non-Asymptotic Analysis of Online Multiplicative Stochastic Gradient Descent Riddhiman Bhattacharya Tiefeng Jiang 54 0 0 14 Dec 2021
Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization Zaiwei Chen Shancong Mou S. T. Maguluri 50 13 0 11 Nov 2021
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 97 4 0 07 Nov 2021
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Margalit Glasgow Honglin Yuan Tengyu Ma FedML 79 45 0 05 Nov 2021
Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers Liam Hodgkinson Umut Simsekli Rajiv Khanna Michael W. Mahoney 77 23 0 02 Aug 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 80 29 0 09 Jun 2021