Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes

26 January 2022

Max Simchowitz

Papers citing "Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes"

43 / 43 papers shown

Title
Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration Avinandan Bose Zhihan Xiong Aadirupa Saha S. Du Maryam Fazel 71 1 0 13 Dec 2024
Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data Chengrui Qu Laixi Shi Kishan Panaganti Pengcheng You Adam Wierman OffRL OnRL 36 0 0 06 Nov 2024
Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learning Zhishuai Liu Weixin Wang Pan Xu 28 1 0 30 Sep 2024
Efficient Reinforcement Learning in Probabilistic Reward Machines Xiaofeng Lin Xuezhou Zhang 54 0 0 19 Aug 2024
Warm-up Free Policy Optimization: Improved Regret in Linear Markov Decision Processes Asaf B. Cassel Aviv A. Rosenberg 35 1 0 03 Jul 2024
Uncertainty-Aware Reward-Free Exploration with General Function Approximation Junkai Zhang Weitong Zhang Dongruo Zhou Q. Gu 49 2 0 24 Jun 2024
Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback Asaf B. Cassel Haipeng Luo Aviv A. Rosenberg Dmitry Sotnikov OffRL 29 3 0 13 May 2024
Experimental Design for Active Transductive Inference in Large Language Models Subhojyoti Mukherjee Anusha Lalitha Aniket Deshmukh Ge Liu Yifei Ma B. Kveton LRM 35 1 0 12 Apr 2024
Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation Zhishuai Liu Pan Xu OOD OffRL 34 8 0 23 Feb 2024
Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints Dan Qiao Yu-Xiang Wang OffRL 22 3 0 02 Feb 2024
Towards Instance-Optimality in Online PAC Reinforcement Learning Aymen Al Marjani Andrea Tirinzoni Emilie Kaufmann OffRL 14 3 0 31 Oct 2023
Towards Optimal Regret in Adversarial Linear MDPs with Bandit Feedback Haolin Liu Chen-Yu Wei Julian Zimmert 22 6 0 17 Oct 2023
Online RL in Linearly $q^π$ -Realizable MDPs Is as Easy as in Linear MDPs If You Learn What to Ignore Gellert Weisz András Gyorgy Csaba Szepesvári OffRL 70 1 0 11 Oct 2023
Rate-Optimal Policy Optimization for Linear Markov Decision Processes Uri Sherman Alon Cohen Tomer Koren Yishay Mansour 33 7 0 28 Aug 2023
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data Ruiqi Zhang Andrea Zanette OffRL OnRL 35 5 0 10 Jul 2023
Optimistic Active Exploration of Dynamical Systems Bhavya Sukhija Lenart Treven Cansu Sancaktar Sebastian Blaes Stelian Coros Andreas Krause 19 17 0 21 Jun 2023
Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RL Qinghua Liu Gellert Weisz András Gyorgy Chi Jin Csaba Szepesvári OffRL 21 8 0 18 May 2023
Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning Gen Li Wenhao Zhan Jason D. Lee Yuejie Chi Yuxin Chen OffRL OnRL 73 12 0 17 May 2023
Uniform-PAC Guarantees for Model-Based RL with Bounded Eluder Dimension Yue Wu Jiafan He Quanquan Gu 11 2 0 15 May 2023
Provably Feedback-Efficient Reinforcement Learning via Active Reward Learning Dingwen Kong Lin F. Yang 23 9 0 18 Apr 2023
Minimax-Optimal Reward-Agnostic Exploration in Reinforcement Learning Gen Li Yuling Yan Yuxin Chen Jianqing Fan OffRL 68 12 0 14 Apr 2023
Optimal Horizon-Free Reward-Free Exploration for Linear Mixture MDPs Junkai Zhang Weitong Zhang Quanquan Gu 21 3 0 17 Mar 2023
Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards Xiang Li Qiang Sun 24 8 0 09 Mar 2023
Finite-sample Guarantees for Nash Q-learning with Linear Function Approximation Pedro Cisneros-Velarde Oluwasanmi Koyejo 18 1 0 01 Mar 2023
Statistical Complexity and Optimal Algorithms for Non-linear Ridge Bandits Nived Rajaraman Yanjun Han Jiantao Jiao Kannan Ramchandran 11 1 0 12 Feb 2023
Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximation Uri Sherman Tomer Koren Yishay Mansour 29 12 0 30 Jan 2023
Leveraging Offline Data in Online Reinforcement Learning Andrew Wagenmaker Aldo Pacchiano OffRL OnRL 27 36 0 09 Nov 2022
Confident Approximate Policy Iteration for Efficient Local Planning in $q^π$ -realizable MDPs Gellert Weisz András Gyorgy Tadashi Kozuno Csaba Szepesvári 12 7 0 27 Oct 2022
Multi-User Reinforcement Learning with Low Rank Rewards Naman Agarwal Prateek Jain S. Kowshik Dheeraj M. Nagaraj Praneeth Netrapalli OffRL 32 1 0 11 Oct 2022
Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation Dan Qiao Yu-Xiang Wang OffRL 61 13 0 03 Oct 2022
Best Policy Identification in Linear MDPs Jerome Taupin Yassir Jedra Alexandre Proutière 36 3 0 11 Aug 2022
Instance-Dependent Near-Optimal Policy Identification in Linear MDPs via Online Experiment Design Andrew Wagenmaker Kevin G. Jamieson OffRL 21 23 0 06 Jul 2022
Active Learning with Safety Constraints Romain Camilleri Andrew Wagenmaker Jamie Morgenstern Lalit P. Jain Kevin G. Jamieson 21 12 0 22 Jun 2022
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL Jinglin Chen Aditya Modi A. Krishnamurthy Nan Jiang Alekh Agarwal 30 25 0 21 Jun 2022
One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning Pedro Cisneros-Velarde Boxiang Lyu Oluwasanmi Koyejo Mladen Kolar OffRL 18 3 0 31 May 2022
Provable Benefits of Representational Transfer in Reinforcement Learning Alekh Agarwal Yuda Song Wen Sun Kaiwen Wang Mengdi Wang Xuezhou Zhang OffRL 21 33 0 29 May 2022
First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach Andrew Wagenmaker Yifang Chen Max Simchowitz S. Du Kevin G. Jamieson 71 36 0 07 Dec 2021
Online Target Q-learning with Reverse Experience Replay: Efficiently finding the Optimal Policy for Linear MDPs Naman Agarwal Syomantak Chaudhuri Prateek Jain Dheeraj M. Nagaraj Praneeth Netrapalli OffRL 34 21 0 16 Oct 2021
Online Sub-Sampling for Reinforcement Learning with General Function Approximation Dingwen Kong Ruslan Salakhutdinov Ruosong Wang Lin F. Yang OffRL 30 1 0 14 Jun 2021
Model-free Representation Learning and Exploration in Low-rank MDPs Aditya Modi Jinglin Chen A. Krishnamurthy Nan Jiang Alekh Agarwal OffRL 98 78 0 14 Feb 2021
Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints Chi Jin Zhuoran Yang Zhaoran Wang OffRL 107 166 0 06 Jan 2021
Reward-Free Exploration for Reinforcement Learning Chi Jin A. Krishnamurthy Max Simchowitz Tiancheng Yu OffRL 104 194 0 07 Feb 2020
Optimism in Reinforcement Learning with Generalized Linear Function Approximation Yining Wang Ruosong Wang S. Du A. Krishnamurthy 127 135 0 09 Dec 2019