Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning

3 February 2019

Papers citing "Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning"

50 / 78 papers shown

Title
$O(1/k)$ Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation Siddharth Chandak 36 0 0 27 Apr 2025
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 36 0 0 15 Apr 2025
A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression Yixuan Zhang Dongyan Yudong Chen Qiaomin Xie 38 0 0 11 Apr 2025
Near-Optimal Sample Complexity for Iterated CVaR Reinforcement Learning with a Generative Model Zilong Deng Simon Khan Shaofeng Zou 59 0 0 11 Mar 2025
Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation: Finite-Time Analysis Siddharth Chandak 50 1 0 18 Jan 2025
The surprising efficiency of temporal difference learning for rare event prediction Xiaoou Cheng Jonathan Weare OffRL 46 0 0 17 Jan 2025
Simplifying Deep Temporal Difference Learning Matteo Gallici Mattie Fellows Benjamin Ellis B. Pou Ivan Masmitja Jakob Foerster Mario Martin OffRL 62 17 0 05 Jul 2024
No Algorithmic Collusion in Two-Player Blindfolded Game with Thompson Sampling Ningyuan Chen Xuefeng Gao Yi Xiong 54 0 0 23 May 2024
Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning Sihan Zeng Thinh T. Doan 56 5 0 15 May 2024
Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning R. Srikant 46 5 0 28 Jan 2024
Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applications Jie Hu Vishwaraj Doshi Do Young Eun 38 4 0 17 Jan 2024
Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise Shaan ul Haque S. Khodadadian S. T. Maguluri 44 11 0 31 Dec 2023
Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications Rajeeva Laxman Karandikar M. Vidyasagar 27 8 0 05 Dec 2023
Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation Guojun Xiong Jian Li 38 13 0 03 Oct 2023
A primal-dual perspective for distributed TD-learning Han-Dong Lim Donghwan Lee 25 1 0 01 Oct 2023
Chained-DP: Can We Recycle Privacy Budget? Jingyi Li Guangjing Huang Liekang Zeng Lin Chen Xu Chen FedML 36 0 0 12 Sep 2023
Finite-Time Analysis of Minimax Q-Learning for Two-Player Zero-Sum Markov Games: Switching System Approach Dong-hwan Lee 23 2 0 09 Jun 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 33 7 0 30 May 2023
First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities Aleksandr Beznosikov S. Samsonov Marina Sheshukova Alexander Gasnikov A. Naumov Eric Moulines 52 14 0 25 May 2023
Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling Nicolò Dal Fabbro A. Mitra George J. Pappas FedML 37 12 0 14 May 2023
n-Step Temporal Difference Learning with Optimal n Lakshmi Mandal S. Bhatnagar 34 2 0 13 Mar 2023
Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation Zhifa Ke Junyu Zhang Zaiwen Wen 24 0 0 25 Feb 2023
Why Target Networks Stabilise Temporal Difference Methods Matt Fellows Matthew Smith Shimon Whiteson OOD AAML 21 7 0 24 Feb 2023
Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity Han Wang A. Mitra Hamed Hassani George J. Pappas James Anderson FedML 31 21 0 04 Feb 2023
Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems Xin Liu Honghao Wei Lei Ying 42 6 0 13 Dec 2022
Finite-Time Error Bounds for Greedy-GQ Yue Wang Yi Zhou Shaofeng Zou 34 1 0 06 Sep 2022
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences Han Shen Tianyi Chen 54 15 0 21 Jun 2022
Exact Formulas for Finite-Time Estimation Errors of Decentralized Temporal Difference Learning with Linear Function Approximation Xing-ming Guo Bin Hu 13 2 0 20 Apr 2022
Target Network and Truncation Overcome The Deadly Triad in $Q$ -Learning Zaiwei Chen John-Paul Clarke S. T. Maguluri 25 19 0 05 Mar 2022
A Small Gain Analysis of Single Timescale Actor Critic Alexander Olshevsky Bahman Gharesifard 33 20 0 04 Mar 2022
Convex Programs and Lyapunov Functions for Reinforcement Learning: A Unified Perspective on the Analysis of Value-Based Methods Xing-ming Guo Bin Hu OffRL 30 3 0 14 Feb 2022
On the Convergence of SARSA with Linear Function Approximation Shangtong Zhang Rémi Tachet des Combes Romain Laroche 23 10 0 14 Feb 2022
Stochastic Gradient Descent with Dependent Data for Offline Reinforcement Learning Jing-rong Dong Xin T. Tong OffRL 35 2 0 06 Feb 2022
Control Theoretic Analysis of Temporal Difference Learning Dong-hwan Lee Do Wan Kim 27 1 0 29 Dec 2021
Accelerated and instance-optimal policy evaluation with linear function approximation Tianjiao Li Guanghui Lan A. Pananjady OffRL 41 13 0 24 Dec 2021
Finite-Time Error Bounds for Distributed Linear Stochastic Approximation Yixuan Lin V. Gupta Ji Liu 32 3 0 24 Nov 2021
Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes Sihan Zeng Thinh T. Doan Justin Romberg 102 17 0 21 Oct 2021
PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method Ziwei Guan Tengyu Xu Yingbin Liang 20 4 0 13 Oct 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 76 97 0 29 Sep 2021
Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning Pratik Ramprasad Yuantong Li Zhuoran Yang Zhaoran Wang W. Sun Guang Cheng OffRL 50 27 0 08 Aug 2021
Concentration of Contractive Stochastic Approximation and Reinforcement Learning Siddharth Chandak Vivek Borkar Parth Dodhia 45 17 0 27 Jun 2021
Analysis of a Target-Based Actor-Critic Algorithm with Linear Function Approximation Anas Barakat Pascal Bianchi Julien Lehmann 32 9 0 14 Jun 2021
Gradient play in stochastic games: stationary points, convergence, and sample complexity Runyu Zhang Zhaolin Ren Na Li 36 43 0 01 Jun 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximation Zaiwei Chen S. Khodadadian S. T. Maguluri OffRL 65 29 0 26 May 2021
On the Linear convergence of Natural Policy Gradient Algorithm S. Khodadadian P. Jhunjhunwala Sushil Mahavir Varma S. T. Maguluri 42 56 0 04 May 2021
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise Thinh T. Doan 16 15 0 04 Apr 2021
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity Shaocong Ma Ziyi Chen Yi Zhou Shaofeng Zou 17 11 0 30 Mar 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm S. Khodadadian Zaiwei Chen S. T. Maguluri CML OffRL 74 26 0 18 Feb 2021
Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis Gen Li Changxiao Cai Ee Yuting Wei Yuejie Chi OffRL 52 75 0 12 Feb 2021
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants Zaiwei Chen S. T. Maguluri Sanjay Shakkottai Karthikeyan Shanmugam OffRL 105 54 0 02 Feb 2021