Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble

1 July 2021

Pieter Abbeel

Papers citing "Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble"

45 / 45 papers shown

Title
What Matters for Batch Online Reinforcement Learning in Robotics? Perry Dong Suvir Mirchandani Dorsa Sadigh Chelsea Finn OffRL 31 0 0 12 May 2025
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning Tianjian Li Daniel Khashabi 55 0 0 05 May 2025
Fine-Tuning without Performance Degradation Han Wang Adam White Martha White OnRL 187 0 0 01 May 2025
Dynamic Action Interpolation: A Universal Approach for Accelerating Reinforcement Learning with Expert Guidance Wenjun Cao 52 0 0 26 Apr 2025
Yes, Q-learning Helps Offline In-Context RL Denis Tarasov Alexander Nikulin Ilya Zisman Albina Klepach Andrei Polubarov Nikita Lyubaykin Alexander Derevyagin Igor Kiselev Vladislav Kurenkov OffRL OnRL 199 0 0 24 Feb 2025
Skill Expansion and Composition in Parameter Space Tenglong Liu J. Li Yinan Zheng Haoyi Niu Yixing Lan Xin Xu Xianyuan Zhan 58 4 0 09 Feb 2025
ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy Yuhui Chen Shuai Tian Shugao Liu Yingting Zhou Haoran Li Dongbin Zhao OffRL 106 1 0 08 Feb 2025
Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework Yulong Hu Tingting Dong Sen Li OffRL OnRL 64 0 0 24 Jan 2025
FORLAPS: An Innovative Data-Driven Reinforcement Learning Approach for Prescriptive Process Monitoring Mostafa Abbasi Maziyar Khadivi Maryam Ahang Patricia Lasserre Yves Lucet H. Najjaran OffRL 43 1 0 17 Jan 2025
Marvel: Accelerating Safe Online Reinforcement Learning with Finetuned Offline Policy Keru Chen Honghao Wei Zhigang Deng Sen Lin OffRL OnRL 94 0 0 31 Dec 2024
Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration Max Wilcoxson Qiyang Li Kevin Frans Sergey Levine SSL OffRL OnRL 57 0 0 23 Oct 2024
Solving Continual Offline RL through Selective Weights Activation on Aligned Spaces Jifeng Hu Sili Huang Li Shen Zhejian Yang Shengchao Hu Shisong Tang H. Chen Yi-Ju Chang Dacheng Tao Lichao Sun OffRL 39 0 0 21 Oct 2024
Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks Yun Qu Boyuan Wang Jianzhun Shao Yuhang Jiang Chen Chen ... Qiang Fu Wei Yang Guang Yang Lanxiao Huang Xiangyang Ji OffRL 54 9 0 20 Aug 2024
FOSP: Fine-tuning Offline Safe Policy through World Models Chenyang Cao Yucheng Xin Silang Wu Longxiang He Zichen Yan Junbo Tan Xueqian Wang OffRL 61 0 0 06 Jul 2024
Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning Mohammadreza Nakhaei Aidan Scannell Joni Pajarinen OffRL 49 1 0 12 Jun 2024
DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays Bo Xia Yilun Kong Yongzhe Chang Bo Yuan Zhiheng Li Xueqian Wang Bin Liang OffRL 50 3 0 05 Jun 2024
Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning Linjiajie Fang Ruoxue Liu Jing Zhang Wenjia Wang Bing-Yi Jing OffRL 56 2 0 31 May 2024
Exclusively Penalized Q-learning for Offline Reinforcement Learning Junghyuk Yeom Yonghyeon Jo Jungmo Kim Sanghyeon Lee Seungyul Han OffRL 40 2 0 23 May 2024
Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning Changhong Wang Xudong Yu Chenjia Bai Qiaosheng Zhang Zhen Wang 40 1 0 12 May 2024
Enhancing Reinforcement Learning Agents with Local Guides Paul Daoudi Bogdan Robu Christophe Prieur Ludovic Dos Santos M. Barlier OnRL 31 3 0 21 Feb 2024
RLIF: Interactive Imitation Learning as Reinforcement Learning Jianlan Luo Perry Dong Yuexiang Zhai Yi Ma Sergey Levine OffRL 30 14 0 21 Nov 2023
Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias Max Sobol Mark Archit Sharma Fahim Tajwar Rafael Rafailov Sergey Levine Chelsea Finn OffRL OnRL 34 1 0 12 Oct 2023
Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning Trevor A. McInroe Adam Jelley Stefano V. Albrecht Amos Storkey OffRL OnRL 25 6 0 09 Oct 2023
PCGPT: Procedural Content Generation via Transformers Sajad Mohaghegh Mohammad Amin Ramezan Dehnavi Golnoosh Abdollahinejad Matin Hashemi ViT 24 2 0 03 Oct 2023
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps Haoyi Niu Tianying Ji Bingqi Liu Haocheng Zhao Xiangyu Zhu Jianying Zheng Pengfei Huang Guyue Zhou Jianming Hu Xianyuan Zhan OffRL OnRL AI4CE 27 6 0 22 Sep 2023
Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning Zhuoran Li Ling Pan Longbo Huang DiffM OffRL 23 7 0 04 Jul 2023
Revisiting the Minimalist Approach to Offline Reinforcement Learning Denis Tarasov Vladislav Kurenkov Alexander Nikulin Sergey Kolesnikov OffRL 33 37 0 16 May 2023
Adaptive Policy Learning for Offline-to-Online Reinforcement Learning Han Zheng Xufang Luo Pengfei Wei Xuan Song Dongsheng Li Jing Jiang OffRL OnRL 18 21 0 14 Mar 2023
Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning Carolin Schmidt Daniele Gammelli Francisco Câmara Pereira Filipe Rodrigues OffRL 14 4 0 28 Feb 2023
Efficient Online Reinforcement Learning with Offline Data Philip J. Ball Laura M. Smith Ilya Kostrikov Sergey Levine OffRL OnRL 34 163 0 06 Feb 2023
Policy Expansion for Bridging Offline-to-Online Reinforcement Learning Haichao Zhang Weiwen Xu Haonan Yu CLL OffRL OnRL 40 62 0 02 Feb 2023
Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble Chong Li OffRL 32 0 0 07 Dec 2022
Hypernetworks for Zero-shot Transfer in Reinforcement Learning S. Rezaei-Shoshtari Charlotte Morissette F. Hogan Gregory Dudek D. Meger OffRL 17 14 0 28 Nov 2022
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size Alexander Nikulin Vladislav Kurenkov Denis Tarasov Dmitry Akimov Sergey Kolesnikov OffRL 33 14 0 20 Nov 2022
Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision Ashvin Nair Brian Zhu Gokul Narayanan Eugen Solowjow Sergey Levine OffRL OnRL 28 15 0 27 Oct 2022
Adaptive Behavior Cloning Regularization for Stable Offline-to-Online Reinforcement Learning Yi Zhao Rinu Boney Alexander Ilin Arno Solin Joni Pajarinen OffRL OnRL 26 39 0 25 Oct 2022
Sustainable Online Reinforcement Learning for Auto-bidding Zhiyu Mou Yusen Huo Rongquan Bai Mingzhou Xie Chuan Yu Jian Xu Bo Zheng OffRL OnRL 34 15 0 13 Oct 2022
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Rameswar Panda Marc G. Bellemare OffRL OnRL 31 63 0 03 Jun 2022
Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space Kuan Fang Patrick Yin Ashvin Nair Sergey Levine OffRL 58 29 0 17 May 2022
How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation Alex X. Lee Coline Devin Jost Tobias Springenberg Yuxiang Zhou Thomas Lampe A. Abdolmaleki Konstantinos Bousmalis OffRL OnRL 21 15 0 06 May 2022
Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning Chenjia Bai Lingxiao Wang Zhuoran Yang Zhihong Deng Animesh Garg Peng Liu Zhaoran Wang OffRL 31 132 0 23 Feb 2022
Reinforcement Learning in Practice: Opportunities and Challenges Yuxi Li OffRL 36 9 0 23 Feb 2022
Value Penalized Q-Learning for Recommender Systems Chengqian Gao Ke Xu Kuangqi Zhou Lanqing Li Xueqian Wang Bo Yuan P. Zhao OffRL 54 20 0 15 Oct 2021
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL Seyed Kamyar Seyed Ghasemipour Dale Schuurmans S. Gu OffRL 209 119 0 21 Jul 2020
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 340 1,960 0 04 May 2020