A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation

6 June 2018

Papers citing "A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation"

50 / 223 papers shown

Title
Bi-Level Policy Optimization with Nyström Hypergradients Arjun Prakash Naicheng He Denizalp Goktas Amy Greenwald 4 0 0 16 May 2025
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation Jiabin Fan Guoqing Luo Michael Bowling Lili Mou OffRL 68 0 0 26 Apr 2025
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 36 0 0 15 Apr 2025
Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation Chenyu Zhang Xu Chen Xuan Di 87 4 0 17 Feb 2025
The surprising efficiency of temporal difference learning for rare event prediction Xiaoou Cheng Jonathan Weare OffRL 43 0 0 17 Jan 2025
Solving Finite-Horizon MDPs via Low-Rank Tensors Sergio Rozada Jose Luis Orejuela Antonio G. Marques 44 0 0 17 Jan 2025
Digital Twin Calibration with Model-Based Reinforcement Learning Hua Zheng Wei Xie I. Ryzhov Keilung Choy 39 0 0 04 Jan 2025
On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representations Guojun Xiong Shufan Wang Daniel Jiang Jian Li FedML 78 1 0 22 Nov 2024
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications Yue Huang Zhaoxian Wu Shiqian Ma Qing Ling 39 1 0 17 Oct 2024
Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way Jeongyeol Kwon Luke Dotson Yudong Chen Qiaomin Xie 36 1 0 16 Oct 2024
On the Hardness of Decentralized Multi-Agent Policy Evaluation under Byzantine Attacks Hairi Minghong Fang Zifan Zhang Alvaro Velasquez Jia Liu AAML 26 1 0 19 Sep 2024
On the Convergence Rates of Federated Q-Learning across Heterogeneous Environments Muxing Wang Pengkun Yang Lili Su FedML 31 1 0 05 Sep 2024
Robust Q-Learning under Corrupted Rewards Sreejeet Maity Aritra Mitra AAML 35 0 0 05 Sep 2024
Finite-Time Analysis of Asynchronous Multi-Agent TD Learning Nicolò Dal Fabbro Arman Adibi Aritra Mitra George J. Pappas 42 1 0 29 Jul 2024
Deflated Dynamics Value Iteration Jongmin Lee Amin Rakhsha Ernest K. Ryu Amir-massoud Farahmand 46 2 0 15 Jul 2024
On Bellman equations for continuous-time policy evaluation I: discretization and approximation Wenlong Mou Yuhua Zhu OffRL 40 2 0 08 Jul 2024
Simplifying Deep Temporal Difference Learning Matteo Gallici Mattie Fellows Benjamin Ellis B. Pou Ivan Masmitja Jakob Foerster Mario Martin OffRL 62 16 0 05 Jul 2024
Learning the Target Network in Function Space Kavosh Asadi Yao Liu Shoham Sabach Ming Yin Rasool Fakoor 43 0 0 03 Jun 2024
Revisiting Step-Size Assumptions in Stochastic Approximation Caio Kalil Lauand Sean P. Meyn 39 1 0 28 May 2024
A CMDP-within-online framework for Meta-Safe Reinforcement Learning Vanshaj Khattar Yuhao Ding Bilgehan Sel Javad Lavaei Ming Jin OffRL 32 12 0 26 May 2024
Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning Shangding Gu Bilgehan Sel Yuhao Ding Lu Wang Qingwei Lin Alois Knoll Ming Jin 42 1 0 26 May 2024
Gaussian Approximation and Multiplier Bootstrap for Polyak-Ruppert Averaged Linear Stochastic Approximation with Applications to TD Learning S. Samsonov Eric Moulines Qi-Man Shao Zhuo-Song Zhang Alexey Naumov 33 4 0 26 May 2024
SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning Shuai Zhang Heshan Devaka Fernando Miao Liu K. Murugesan Songtao Lu Pin-Yu Chen Tianyi Chen Meng Wang 54 1 0 24 May 2024
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks Zhifa Ke Zaiwen Wen Junyu Zhang 37 0 0 07 May 2024
A Single Online Agent Can Efficiently Learn Mean Field Games Chenyu Zhang Xu Chen Xuan Di OffRL 47 2 0 05 May 2024
An MRP Formulation for Supervised Learning: Generalized Temporal Difference Learning Models Yangchen Pan Junfeng Wen Chenjun Xiao Philip Torr OffRL MU 29 0 0 23 Apr 2024
EMC $^2$ : Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence Chung-Yiu Yau Hoi-To Wai Parameswaran Raman Soumajyoti Sarkar Mingyi Hong 39 1 0 16 Apr 2024
Compressed Federated Reinforcement Learning with a Generative Model Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 39 2 0 26 Mar 2024
DASA: Delay-Adaptive Multi-Agent Stochastic Approximation Nicolò Dal Fabbro Arman Adibi H. Vincent Poor Sanjeev R. Kulkarni A. Mitra George J. Pappas 39 2 0 25 Mar 2024
Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD update Fnu Hairi Zifan Zhang Jia-Wei Liu 20 2 0 23 Mar 2024
One-Shot Averaging for Distributed TD( $λ$ ) Under Markov Sampling Haoxing Tian I. Paschalidis Alexander Olshevsky OffRL 47 4 0 13 Mar 2024
Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation Zhishuai Liu Pan Xu OOD OffRL 42 8 0 23 Feb 2024
Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling Arman Adibi Nicolò Dal Fabbro Luca Schenato Sanjeev R. Kulkarni H. Vincent Poor George J. Pappas Hamed Hassani A. Mitra 35 8 0 19 Feb 2024
SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning Paul Mangold S. Samsonov Safwan Labbi I. Levin Réda Alami Alexey Naumov Eric Moulines 38 1 0 06 Feb 2024
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff 45 1 0 05 Feb 2024
Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems Neharika Jali Guannan Qu Weina Wang Gauri Joshi 29 5 0 02 Feb 2024
Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning Chenyu Zhang Han Wang Aritra Mitra James Anderson 37 18 0 27 Jan 2024
Regularized Q-Learning with Linear Function Approximation Jiachen Xi Alfredo Garcia P. Momcilovic 38 2 0 26 Jan 2024
Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation Yixuan Zhang Qiaomin Xie 35 4 0 25 Jan 2024
Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving $O(1/k)$ Finite-Sample Complexity Thinh T. Doan 32 7 0 23 Jan 2024
Stochastic optimization with arbitrary recurrent data sampling William G. Powell Hanbaek Lyu 37 0 0 15 Jan 2024
Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise Shaan ul Haque S. Khodadadian S. T. Maguluri 44 11 0 31 Dec 2023
Prediction and Control in Continual Reinforcement Learning N. Anand Doina Precup OffRL CLL 32 11 0 18 Dec 2023
Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference D. Huo Yudong Chen Qiaomin Xie 34 4 0 18 Dec 2023
A Concentration Bound for TD(0) with Function Approximation Siddharth Chandak Vivek Borkar 31 0 0 16 Dec 2023
On the Performance of Temporal Difference Learning With Neural Networks Haoxing Tian I. Paschalidis Alexander Olshevsky 24 5 0 08 Dec 2023
Finite-Time Analysis of Three-Timescale Constrained Actor-Critic and Constrained Natural Actor-Critic Algorithms Prashansa Panda Shalabh Bhatnagar 41 0 0 25 Oct 2023
On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$ -Greedy Exploration Shuai Zhang Hongkang Li Meng Wang Miao Liu Pin-Yu Chen Songtao Lu Sijia Liu K. Murugesan Subhajit Chaudhury 40 19 0 24 Oct 2023
Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation Guojun Xiong Jian Li 35 13 0 03 Oct 2023
A primal-dual perspective for distributed TD-learning Han-Dong Lim Donghwan Lee 25 1 0 01 Oct 2023