Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis

12 February 2021

Papers citing "Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis"

50 / 53 papers shown

Title
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 36 0 0 15 Apr 2025
Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation Chenyu Zhang Xu Chen Xuan Di 81 4 0 17 Feb 2025
On the Convergence Rates of Federated Q-Learning across Heterogeneous Environments Muxing Wang Pengkun Yang Lili Su FedML 17 1 0 05 Sep 2024
Robust Q-Learning under Corrupted Rewards Sreejeet Maity Aritra Mitra AAML 18 0 0 05 Sep 2024
Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Control Zifan Liu Xinran Li Shibo Chen Gen Li Jiashuo Jiang Jun Zhang 25 0 0 26 Jun 2024
A finite time analysis of distributed Q-learning Han-Dong Lim Donghwan Lee OffRL 32 0 0 23 May 2024
Federated Control in Markov Decision Processes Hao Jin Yang Peng Liangyu Zhang Zhihua Zhang FedML 25 0 0 07 May 2024
A Single Online Agent Can Efficiently Learn Mean Field Games Chenyu Zhang Xu Chen Xuan Di OffRL 28 2 0 05 May 2024
Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm Miao Lu Han Zhong Tong Zhang Jose H. Blanchet OffRL OOD 68 4 0 04 Apr 2024
Compressed Federated Reinforcement Learning with a Generative Model Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 30 2 0 26 Mar 2024
A Natural Extension To Online Algorithms For Hybrid RL With Limited Coverage Kevin Tan Ziping Xu OffRL OnRL 24 4 0 07 Mar 2024
Finite-Time Error Analysis of Online Model-Based Q-Learning with a Relaxed Sampling Model Han-Dong Lim HyeAnn Lee Donghwan Lee OffRL 20 0 0 19 Feb 2024
Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices Jiin Woo Laixi Shi Gauri Joshi Yuejie Chi OffRL 24 3 0 08 Feb 2024
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation Yixuan Zhang Qiaomin Xie 16 4 0 25 Jan 2024
A Concentration Bound for TD(0) with Function Approximation Siddharth Chandak Vivek Borkar 16 0 0 16 Dec 2023
Optimal Sample Complexity for Average Reward Markov Decision Processes Shengbo Wang Jose H. Blanchet Peter Glynn 15 8 0 13 Oct 2023
Minimax Optimal Q Learning with Nearest Neighbors Puning Zhao Lifeng Lai OffRL 43 10 0 03 Aug 2023
Settling the Sample Complexity of Online Reinforcement Learning Zihan Zhang Yuxin Chen Jason D. Lee S. Du OffRL 90 21 0 25 Jul 2023
Sharper Model-free Reinforcement Learning for Average-reward Markov Decision Processes Zihan Zhang Qiaomin Xie OffRL 11 16 0 28 Jun 2023
Achieving Sample and Computational Efficient Reinforcement Learning by Action Space Reduction via Grouping Yining Li Peizhong Ju Ness B. Shroff 14 0 0 22 Jun 2023
Off-policy Evaluation in Doubly Inhomogeneous Environments Zeyu Bian C. Shi Zhengling Qi Lan Wang OffRL 21 3 0 14 Jun 2023
Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds Jiayi Huang Han Zhong Liwei Wang Lin F. Yang 20 6 0 12 Jun 2023
High-probability sample complexities for policy evaluation with linear function approximation Gen Li Weichen Wu Yuejie Chi Cong Ma Alessandro Rinaldo Yuting Wei OffRL 18 6 0 30 May 2023
Sample Complexity of Variance-reduced Distributionally Robust Q-learning Shengbo Wang Nian Si Jose H. Blanchet Zhengyuan Zhou OOD 13 12 0 28 May 2023
The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model Laixi Shi Gen Li Yuting Wei Yuxin Chen M. Geist Yuejie Chi OOD 25 23 0 26 May 2023
The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond Jiin Woo Gauri Joshi Yuejie Chi FedML 14 19 0 18 May 2023
Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards Xiang Li Qiang Sun 16 8 0 09 Mar 2023
On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples Mustafa O. Karabag Ufuk Topcu OffRL 29 4 0 07 Mar 2023
A Finite Sample Complexity Bound for Distributionally Robust Q-learning Shengbo Wang Nian Si Jose H. Blanchet Zhengyuan Zhou OOD OffRL 21 22 0 26 Feb 2023
Optimal Sample Complexity of Reinforcement Learning for Mixing Discounted Markov Decision Processes Shengbo Wang Jose H. Blanchet Peter Glynn 18 4 0 15 Feb 2023
Minimax-Optimal Multi-Agent RL in Markov Games With a Generative Model Gen Li Yuejie Chi Yuting Wei Yuxin Chen 17 18 0 22 Aug 2022
Towards Global Optimality in Cooperative MARL with the Transformation And Distillation Framework Jianing Ye Chenghao Li Jianhao Wang Chongjie Zhang 30 2 0 12 Jul 2022
Overcoming the Long Horizon Barrier for Sample-Efficient Reinforcement Learning with Latent Low-Rank Structure Tyler Sam Yudong Chen C. Yu OffRL 18 6 0 07 Jun 2022
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning Andrea Zanette Martin J. Wainwright OOD 19 5 0 01 Jun 2022
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal Tadashi Kozuno Wenhao Yang Nino Vieillard Toshinori Kitamura Yunhao Tang ... Michal Valko Rémi Munos Olivier Pietquin M. Geist Csaba Szepesvári 85 10 0 27 May 2022
JUNO: Jump-Start Reinforcement Learning-based Node Selection for UWB Indoor Localization Zohreh Hajiakhondi-Meybodi Ming Hou Arash Mohammadi 15 3 0 06 May 2022
A Note on Target Q-learning For Solving Finite MDPs with A Generative Oracle Ziniu Li Tian Xu Yang Yu 26 5 0 22 Mar 2022
The Efficacy of Pessimism in Asynchronous Q-Learning Yuling Yan Gen Li Yuxin Chen Jianqing Fan OffRL 68 40 0 14 Mar 2022
Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity Laixi Shi Gen Li Yuting Wei Yuxin Chen Yuejie Chi OffRL 11 90 0 28 Feb 2022
Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons C. Shi S. Luo Yuan Le Hongtu Zhu R. Song OffRL OnRL 11 10 0 26 Feb 2022
Optimal variance-reduced stochastic approximation in Banach spaces Wenlong Mou K. Khamaru Martin J. Wainwright Peter L. Bartlett Michael I. Jordan 16 8 0 21 Jan 2022
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning Xiang Li Wenhao Yang Jiadong Liang Zhihua Zhang Michael I. Jordan 27 15 0 29 Dec 2021
Accelerated and instance-optimal policy evaluation with linear function approximation Tianjiao Li Guanghui Lan A. Pananjady OffRL 17 13 0 24 Dec 2021
Convergence Results For Q-Learning With Experience Replay Liran Szlak Ohad Shamir OffRL 11 5 0 08 Dec 2021
A Concentration Bound for LSPE( $λ$ ) Siddharth Chandak Vivek Borkar H. Dolhare 25 0 0 04 Nov 2021
Online Target Q-learning with Reverse Experience Replay: Efficiently finding the Optimal Policy for Linear MDPs Naman Agarwal Syomantak Chaudhuri Prateek Jain Dheeraj M. Nagaraj Praneeth Netrapalli OffRL 34 21 0 16 Oct 2021
Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning Gen Li Laixi Shi Yuxin Chen Yuejie Chi OffRL 23 50 0 09 Oct 2021
Online Robust Reinforcement Learning with Model Uncertainty Yue Wang Shaofeng Zou OOD OffRL 68 96 0 29 Sep 2021
Concentration of Contractive Stochastic Approximation and Reinforcement Learning Siddharth Chandak Vivek Borkar Parth Dodhia 26 17 0 27 Jun 2021
Navigating to the Best Policy in Markov Decision Processes Aymen Al Marjani Aurélien Garivier Alexandre Proutière 14 20 0 05 Jun 2021