v1v2 (latest)

Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation

Annual Conference Computational Learning Theory (COLT), 2021

21 November 2021

Papers citing "Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation"

50 / 53 papers shown

Title
Finite-Time Bounds for Average-Reward Fitted Q-Iteration Jongmin Lee Ernest K. Ryu OffRL 76 0 0 20 Oct 2025
Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Finite-Horizon Offline RL with Linear $q^π$ -Realizability and Concentrability Volodymyr Tkachuk Csaba Szepesvári Xiaoqi Tan OffRL 84 0 0 03 Oct 2025
Inverse Reinforcement Learning Using Just Classification and a Few Regressions Lars van der Laan Nathan Kallus Aurélien F. Bibaut 68 0 0 25 Sep 2025
A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory Fengdi Che OffRL 116 0 0 11 Aug 2025
The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportability Jiachen Hu Rui Ai Han Zhong Xiaoyu Chen L. Wang Zhaoran Wang Zhuoran Yang 181 0 0 11 Jun 2025
On The Statistical Complexity of Offline Decision-MakingInternational Conference on Machine Learning (ICML), 2025 Thanh Nguyen-Tang R. Arora OffRL 408 2 0 10 Jan 2025
Primal-Dual Spectral Representation for Off-policy EvaluationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Yang Hu Tianyi Chen Na Li Kai Wang Bo Dai OffRL 258 3 0 23 Oct 2024
The Central Role of the Loss Function in Reinforcement Learning Kaiwen Wang Nathan Kallus Wen Sun OffRL 660 10 0 19 Sep 2024
The Role of Inherent Bellman Error in Offline Reinforcement Learning with Linear Function Approximation Noah Golowich Ankur Moitra OffRL 277 3 0 17 Jun 2024
Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear $q^π$ -Realizability and Concentrability Volodymyr Tkachuk Gellert Weisz Csaba Szepesvári OffRL 169 3 0 27 May 2024
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL Yifei Zhou Andrea Zanette Jiayi Pan Sergey Levine Aviral Kumar 285 120 0 29 Feb 2024
Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization Philip Ndikum Serge Ndikum 232 7 0 27 Feb 2024
Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement LearningAnnual Conference Computational Learning Theory (COLT), 2024 Philip Amortila Tongyi Cao Akshay Krishnamurthy OffRL OOD 208 4 0 22 Jan 2024
Offline Data Enhanced On-Policy Policy Gradient with Provable GuaranteesInternational Conference on Learning Representations (ICLR), 2023 Yifei Zhou Ayush Sekhari Yuda Song Wen Sun OffRL OnRL 200 8 0 14 Nov 2023
On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics Michal Nauman Marek Cygan 363 5 0 30 Oct 2023
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and AutoregressionInternational Conference on Learning Representations (ICLR), 2023 Adam Block Dylan J. Foster Akshay Krishnamurthy Max Simchowitz Cyril Zhang 235 9 0 17 Oct 2023
When is Agnostic Reinforcement Learning Statistically Tractable?Neural Information Processing Systems (NeurIPS), 2023 Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 268 7 0 09 Oct 2023
The Optimal Approximation Factors in Misspecified Off-Policy Value Function EstimationInternational Conference on Machine Learning (ICML), 2023 Philip Amortila Nan Jiang Csaba Szepesvári OffRL 224 4 0 25 Jul 2023
Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems Xiang Ji Huazheng Wang Minshuo Chen Tuo Zhao Mengdi Wang OffRL 278 8 0 24 Jul 2023
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline DataNeural Information Processing Systems (NeurIPS), 2023 Ruiqi Zhang Andrea Zanette OffRL OnRL 248 9 0 10 Jul 2023
Active Coverage for PAC Reinforcement LearningAnnual Conference Computational Learning Theory (COLT), 2023 Aymen Al Marjani Andrea Tirinzoni E. Kaufmann OffRL 171 5 0 23 Jun 2023
Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory WeightingInternational Conference on Learning Representations (ICLR), 2023 Zhang-Wei Hong Pulkit Agrawal Rémi Tachet des Combes Romain Laroche OffRL 178 25 0 22 Jun 2023
A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement LearningInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Kihyuk Hong Yuhang Li Ambuj Tewari OffRL 306 9 0 13 Jun 2023
Unified Off-Policy Learning to Rank: a Reinforcement Learning PerspectiveNeural Information Processing Systems (NeurIPS), 2023 Zeyu Zhang Yi-Hsun Su Hui Yuan Yiran Wu R. Balasubramanian Qingyun Wu Huazheng Wang Mengdi Wang OffRL CML 340 7 0 13 Jun 2023
Survival Instinct in Offline Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2023 Anqi Li Dipendra Kumar Misra Andrey Kolobov Ching-An Cheng OffRL 212 20 0 05 Jun 2023
The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2023 Kaiwen Wang Kevin Zhou Runzhe Wu Nathan Kallus Wen Sun OffRL 422 23 0 25 May 2023
Offline Reinforcement Learning with Additional Covering Distributions Chenjie Mao OffRL 223 0 0 22 May 2023
Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions Yicheng Luo Jackie Kay Edward Grefenstette M. Deisenroth OffRL OnRL 165 20 0 30 Mar 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function ApproximationInternational Conference on Learning Representations (ICLR), 2023 Thanh Nguyen-Tang R. Arora OffRL 194 6 0 24 Feb 2023
Minimax Instrumental Variable Regression and $L_2$ Convergence Guarantees without Identification or ClosednessAnnual Conference Computational Learning Theory (COLT), 2023 Andrew Bennett Nathan Kallus Xiaojie Mao Whitney Newey Vasilis Syrgkanis Masatoshi Uehara 227 17 0 10 Feb 2023
Selective Uncertainty Propagation in Offline RLAAAI Conference on Artificial Intelligence (AAAI), 2023 Sanath Kumar Krishnamurthy Shrey Modi Tanmay Gangwani S. Katariya Branislav Kveton A. Rangi OffRL 531 0 0 01 Feb 2023
Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling Ashish Kumar Ilya Kuzovkin OffRL OnRL 130 2 0 16 Dec 2022
A Review of Off-Policy Evaluation in Reinforcement Learning Masatoshi Uehara C. Shi Nathan Kallus OffRL 226 99 0 13 Dec 2022
When is Realizability Sufficient for Off-Policy Reinforcement Learning?International Conference on Machine Learning (ICML), 2022 Andrea Zanette OffRL 234 16 0 10 Nov 2022
Oracle Inequalities for Model Selection in Offline Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2022 Jonathan Lee George Tucker Ofir Nachum Bo Dai Emma Brunskill OffRL 310 14 0 03 Nov 2022
Behavior Prior Representation learning for Offline Reinforcement LearningInternational Conference on Learning Representations (ICLR), 2022 Hongyu Zang Xin Li Jie Yu Chen Liu Riashat Islam Rémi Tachet des Combes Romain Laroche OffRL OnRL 305 12 0 02 Nov 2022
Optimal Conservative Offline RL with General Function Approximation via Augmented LagrangianInternational Conference on Learning Representations (ICLR), 2022 Paria Rashidinejad Hanlin Zhu Kunhe Yang Stuart J. Russell Jiantao Jiao OffRL 344 32 0 01 Nov 2022
Agent-Controller Representations: Principled Offline RL with Rich Exogenous InformationInternational Conference on Machine Learning (ICML), 2022 Riashat Islam Manan Tomar Alex Lamb Yonathan Efroni Hongyu Zang ... Dipendra Kumar Misra Xin-hui Li H. V. Seijen Rémi Tachet des Combes John Langford OffRL 212 10 0 31 Oct 2022
Hybrid RL: Using Both Offline and Online Data Can Make RL EfficientInternational Conference on Learning Representations (ICLR), 2022 Yuda Song Yi Zhou Ayush Sekhari J. Andrew Bagnell A. Krishnamurthy Wen Sun OffRL OnRL 311 132 0 13 Oct 2022
Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning Tung Nguyen Qinqing Zheng Aditya Grover OffRL 281 7 0 11 Oct 2022
The Role of Coverage in Online Reinforcement LearningInternational Conference on Learning Representations (ICLR), 2022 Tengyang Xie Dylan J. Foster Yu Bai Nan Jiang Sham Kakade OffRL 232 69 0 09 Oct 2022
Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient Ming Yin Mengdi Wang Yu Wang OffRL 269 12 0 03 Oct 2022
Learning Bellman Complete Representations for Offline Policy EvaluationInternational Conference on Machine Learning (ICML), 2022 Jonathan D. Chang Kaiwen Wang Nathan Kallus Wen Sun OffRL 179 17 0 12 Jul 2022
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RLNeural Information Processing Systems (NeurIPS), 2022 Jinglin Chen Aditya Modi A. Krishnamurthy Nan Jiang Alekh Agarwal 291 28 0 21 Jun 2022
Provably Efficient Offline Reinforcement Learning with Trajectory-Wise RewardIEEE Transactions on Information Theory (IEEE Trans. Inf. Theory), 2022 Tengyu Xu Yue Wang Shaofeng Zou Yingbin Liang OffRL 198 15 0 13 Jun 2022
Stabilizing Q-learning with Linear Architectures for Provably Efficient LearningInternational Conference on Machine Learning (ICML), 2022 Andrea Zanette Martin J. Wainwright OOD 266 5 0 01 Jun 2022
$Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets$ Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence SetsNeural Information Processing Systems (NeurIPS), 2022 Gen Li Cong Ma Nathan Srebro OffRL 250 18 0 21 May 2022
Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of GapsConference on Uncertainty in Artificial Intelligence (UAI), 2022 Jinglin Chen Nan Jiang OffRL 303 36 0 25 Mar 2022
Bellman Residual Orthogonalization for Offline Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2022 Andrea Zanette Martin J. Wainwright OffRL 302 8 0 24 Mar 2022
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with PessimismInternational Conference on Learning Representations (ICLR), 2022 Ming Yin Yaqi Duan Mengdi Wang Yu Wang OffRL 228 68 0 11 Mar 2022