Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2102.01567
Cited By
v1
v2
v3
v4 (latest)
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants
2 February 2021
Zaiwei Chen
S. T. Maguluri
Sanjay Shakkottai
Karthikeyan Shanmugam
OffRL
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants"
38 / 38 papers shown
Central Limit Theorems for Asynchronous Averaged Q-Learning
Xingtu Liu
241
1
0
23 Sep 2025
Statistical and Algorithmic Foundations of Reinforcement Learning
Yuejie Chi
Yuxin Chen
Yuting Wei
OffRL
278
3
0
19 Jul 2025
Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features
Zixuan Xie
Xinyu Liu
Rohan Chandra
Shangtong Zhang
450
3
0
27 May 2025
Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning
Emile Anand
Ishani Karmarkar
Guannan Qu
735
8
0
01 Dec 2024
A finite time analysis of distributed Q-learning
Han-Dong Lim
Donghwan Lee
OffRL
422
1
0
23 May 2024
Is Thompson Sampling Susceptible to Algorithmic Collusion?
Yi Xiong
Ningyuan Chen
Yi Xiong
360
0
0
23 May 2024
Compressed Federated Reinforcement Learning with a Generative Model
Ali Beikmohammadi
Sarit Khirirat
Sindri Magnússon
FedML
396
5
0
26 Mar 2024
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
Yixuan Zhang
Qiaomin Xie
350
16
0
25 Jan 2024
A Concentration Bound for TD(0) with Function Approximation
Siddharth Chandak
Vivek Borkar
566
4
0
16 Dec 2023
Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications
Journal of Optimization Theory and Applications (JOTA), 2023
Rajeeva Laxman Karandikar
M. Vidyasagar
501
23
0
05 Dec 2023
Finite-Time Analysis of Minimax Q-Learning for Two-Player Zero-Sum Markov Games: Switching System Approach
Dong-hwan Lee
297
4
0
09 Jun 2023
The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond
International Conference on Machine Learning (ICML), 2023
Jiin Woo
Gauri Joshi
Yuejie Chi
FedML
402
34
0
18 May 2023
Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise
Zaiwei Chen
S. T. Maguluri
Martin Zubeldia
301
25
0
28 Mar 2023
Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games
Conference on Uncertainty in Artificial Intelligence (UAI), 2023
Zhaoyi Zhou
Zaiwei Chen
Yiheng Lin
Adam Wierman
368
9
0
08 Mar 2023
A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Games
Neural Information Processing Systems (NeurIPS), 2023
Zaiwei Chen
Jianchao Tan
Eric Mazumdar
Asuman Ozdaglar
Adam Wierman
383
16
0
03 Mar 2023
Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
Measurement and Modeling of Computer Systems (SIGMETRICS), 2022
D. Huo
Yudong Chen
Qiaomin Xie
321
23
0
03 Oct 2022
First-order Policy Optimization for Robust Markov Decision Process
Yan Li
Guanghui Lan
Tuo Zhao
510
34
0
21 Sep 2022
An Approximate Policy Iteration Viewpoint of Actor-Critic Algorithms
Zaiwei Chen
S. T. Maguluri
224
2
0
05 Aug 2022
Finite-Time Analysis of Asynchronous Q-learning under Diminishing Step-Size from Control-Theoretic View
IEEE Access (IEEE Access), 2022
Han-Dong Lim
Dong-hwan Lee
158
3
0
25 Jul 2022
The Efficacy of Pessimism in Asynchronous Q-Learning
IEEE Transactions on Information Theory (IEEE Trans. Inf. Theory), 2022
Yuling Yan
Gen Li
Yuxin Chen
Jianqing Fan
OffRL
397
45
0
14 Mar 2022
Target Network and Truncation Overcome The Deadly Triad in
Q
Q
Q
-Learning
SIAM Journal on Mathematics of Data Science (SIMODS), 2022
Zaiwei Chen
John-Paul Clarke
S. T. Maguluri
309
31
0
05 Mar 2022
On the Convergence of SARSA with Linear Function Approximation
International Conference on Machine Learning (ICML), 2022
Shangtong Zhang
Rémi Tachet des Combes
Romain Laroche
281
18
0
14 Feb 2022
Stochastic Gradient Descent with Dependent Data for Offline Reinforcement Learning
Jing-rong Dong
Xin T. Tong
OffRL
277
2
0
06 Feb 2022
Optimal variance-reduced stochastic approximation in Banach spaces
Wenlong Mou
K. Khamaru
Martin J. Wainwright
Peter L. Bartlett
Sai Li
292
11
0
21 Jan 2022
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning
International Conference on Artificial Intelligence and Statistics (AISTATS), 2021
Xiang Li
Wenhao Yang
Jiadong Liang
Zhihua Zhang
Michael I. Jordan
457
26
0
29 Dec 2021
Accelerated and instance-optimal policy evaluation with linear function approximation
SIAM Journal on Mathematics of Data Science (SIMODS), 2021
Tianjiao Li
Guanghui Lan
A. Pananjady
OffRL
259
18
0
24 Dec 2021
A Concentration Bound for LSPE(
λ
λ
λ
)
Social Science Research Network (SSRN), 2021
Siddharth Chandak
Vivek Borkar
H. Dolhare
416
0
0
04 Nov 2021
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
Journal of machine learning research (JMLR), 2021
Shangtong Zhang
Rémi Tachet des Combes
Romain Laroche
519
18
0
04 Nov 2021
Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning
Neural Information Processing Systems (NeurIPS), 2021
Gen Li
Laixi Shi
Yuxin Chen
Yuejie Chi
OffRL
402
66
0
09 Oct 2021
Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning
Communications in Optimization Theory (COT), 2021
Rajeeva Laxman Karandikar
M. Vidyasagar
309
0
0
08 Sep 2021
Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning
Pratik Ramprasad
Yuantong Li
Zhuoran Yang
Zhaoran Wang
W. Sun
Guang Cheng
OffRL
504
40
0
08 Aug 2021
Concentration of Contractive Stochastic Approximation and Reinforcement Learning
Stochastic Systems (SS), 2021
Siddharth Chandak
Vivek Borkar
Parth Dodhia
353
30
0
27 Jun 2021
Finite-Sample Analysis of Off-Policy TD-Learning via Generalized Bellman Operators
Zaiwei Chen
S. T. Maguluri
Sanjay Shakkottai
Karthikeyan Shanmugam
OffRL
209
20
0
24 Jun 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximation
IEEE Control Systems Letters (L-CSS), 2021
Zaiwei Chen
S. Khodadadian
S. T. Maguluri
OffRL
301
33
0
26 May 2021
On the Linear convergence of Natural Policy Gradient Algorithm
IEEE Conference on Decision and Control (CDC), 2021
S. Khodadadian
P. Jhunjhunwala
Sushil Mahavir Varma
S. T. Maguluri
369
66
0
04 May 2021
Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm
International Conference on Machine Learning (ICML), 2021
S. Khodadadian
Zaiwei Chen
S. T. Maguluri
CML
OffRL
369
33
0
18 Feb 2021
A Discrete-Time Switching System Analysis of Q-learning
SIAM Journal of Control and Optimization (SICON), 2021
Donghwan Lee
Jianghai Hu
Niao He
535
20
0
17 Feb 2021
Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis
Operational Research (OR), 2021
Gen Li
Changxiao Cai
Ee
Yuting Wei
Yuejie Chi
OffRL
437
96
0
12 Feb 2021
1
Page 1 of 1
