DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization

9 December 2021

Aaron Courville

Papers citing "DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization"

46 / 46 papers shown

Title
FAST-Q: Fast-track Exploration with Adversarially Balanced State Representations for Counterfactual Action Estimation in Offline Reinforcement Learning Pulkit Agrawal Rukma Talwadker Aditya Pareek Tridib Mukherjee OffRL 30 0 0 30 Apr 2025
Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning Mingqi Yuan Qi Wang Guozheng Ma Bo-wen Li Xin Jin Yunbo Wang Xiaokang Yang Wenjun Zeng D. Tao OffRL AI4CE 33 0 0 24 Apr 2025
Digi-Q: Learning Q-Value Functions for Training Device-Control Agents Hao Bai Yifei Zhou Li Erran Li Sergey Levine Aviral Kumar OffRL 45 1 0 13 Feb 2025
Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrations Bryan Chan Anson Leung James Bergstra OffRL OnRL 52 0 0 19 Oct 2024
SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning Hojoon Lee Dongyoon Hwang Donghu Kim Hyunseung Kim Jun Jet Tai K. Subramanian Peter R. Wurman Jaegul Choo Peter Stone Takuma Seno OffRL 62 6 0 13 Oct 2024
Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn Hongyao Tang Glen Berseth OffRL 40 1 0 07 Sep 2024
Unsupervised-to-Online Reinforcement Learning Junsu Kim Seohong Park Sergey Levine OnRL 48 3 0 27 Aug 2024
ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning Hosung Lee Sejin Kim Seungpil Lee Sanha Hwang Jihwan Lee Byung-Jun Lee Sundong Kim LRM 37 8 0 30 Jul 2024
Is Value Learning Really the Main Bottleneck in Offline RL? Seohong Park Kevin Frans Sergey Levine Aviral Kumar OffRL 45 7 0 13 Jun 2024
A Dual Approach to Imitation Learning from Observations with Offline Datasets Harshit S. Sikchi Caleb Chuck Amy Zhang S. Niekum OffRL 25 4 0 13 Jun 2024
Is Value Functions Estimation with Classification Plug-and-play for Offline Reinforcement Learning? Denis Tarasov Kirill Brilliantov Dmitrii Kharlapenko OffRL 30 2 0 10 Jun 2024
Which Experiences Are Influential for RL Agents? Efficiently Estimating The Influence of Experiences Takuya Hiraoka Guanquan Wang Takashi Onishi Yoshimasa Tsuruoka 37 0 0 23 May 2024
Offline Reinforcement Learning from Datasets with Structured Non-Stationarity Johannes Ackermann Takayuki Osa Masashi Sugiyama OffRL 37 2 0 23 May 2024
The Curse of Diversity in Ensemble-Based Exploration Zhixuan Lin P. DÓro Evgenii Nikishin Aaron C. Courville 40 1 0 07 May 2024
Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning Caleb Chuck Carl Qi M. Munje Shuozhe Li Max Rudolph ... Kavan Mehta Anthony Wang Peter Stone Amy Zhang S. Niekum 38 5 0 06 May 2024
No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO Skander Moalla Andrea Miele Razvan Pascanu Çağlar Gülçehre 26 5 0 01 May 2024
Adaptive Regularization of Representation Rank as an Implicit Constraint of Bellman Equation Qiang He Tianyi Zhou Meng Fang S. Maghsudi 27 3 0 19 Apr 2024
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL Jesse Farebrother Jordi Orbay Q. Vuong Adrien Ali Taïga Yevgen Chebotar ... Sergey Levine Pablo Samuel Castro Aleksandra Faust Aviral Kumar Rishabh Agarwal OffRL 56 56 0 06 Mar 2024
In value-based deep reinforcement learning, a pruned network is a good network J. Obando-Ceron Aaron C. Courville Pablo Samuel Castro OffRL 36 18 0 19 Feb 2024
Mixtures of Experts Unlock Parameter Scaling for Deep RL J. Obando-Ceron Ghada Sokar Timon Willi Clare Lyle Jesse Farebrother Jakob N. Foerster Gintare Karolina Dziugaite Doina Precup Pablo Samuel Castro 50 29 0 13 Feb 2024
ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update Liyuan Mao Haoran Xu Weinan Zhang Xianyuan Zhan 27 10 0 01 Feb 2024
Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms Pengyi Li Jianye Hao Hongyao Tang Xian Fu Yan Zheng Ke Tang 29 9 0 22 Jan 2024
Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL Yang Yue Rui Lu Bingyi Kang Shiji Song Gao Huang OffRL 27 16 0 06 Oct 2023
Small batch deep reinforcement learning J. Obando-Ceron Marc G. Bellemare Pablo Samuel Castro VLM 32 14 0 05 Oct 2023
HIQL: Offline Goal-Conditioned RL with Latent States as Actions Seohong Park Dibya Ghosh Benjamin Eysenbach Sergey Levine OffRL 30 44 0 22 Jul 2023
JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning Kaiwen Wang Junxiong Wang Yueying Li Nathan Kallus Immanuel Trummer Wen Sun GP 42 2 0 21 Jul 2023
Deep Generative Models for Decision-Making and Control Michael Janner 29 1 0 15 Jun 2023
HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach Shixi Lian Yi-An Ma Jinyi Liu Yan Zheng Zhaopeng Meng OffRL 16 1 0 10 Jun 2023
Efficient Deep Reinforcement Learning Requires Regulating Overfitting Qiyang Li Aviral Kumar Ilya Kostrikov Sergey Levine OffRL 24 30 0 20 Apr 2023
The Ladder in Chaos: A Simple and Effective Improvement to General DRL Algorithms by Policy Path Trimming and Boosting Hongyao Tang M. Zhang Jianye Hao 23 1 0 02 Mar 2023
The Dormant Neuron Phenomenon in Deep Reinforcement Learning Ghada Sokar Rishabh Agarwal P. S. Castro Utku Evci CLL 40 88 0 24 Feb 2023
Which Experiences Are Influential for Your Agent? Policy Iteration with Turn-over Dropout Takuya Hiraoka Takashi Onishi Yoshimasa Tsuruoka OffRL 19 0 0 26 Jan 2023
Confidence-Conditioned Value Functions for Offline Reinforcement Learning Joey Hong Aviral Kumar Sergey Levine OffRL 25 20 0 08 Dec 2022
Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes Aviral Kumar Rishabh Agarwal Xinyang Geng George Tucker Sergey Levine OffRL 39 48 0 28 Nov 2022
Behavior Prior Representation learning for Offline Reinforcement Learning Hongyu Zang Xin Li Jie Yu Chen Liu Riashat Islam Rémi Tachet des Combes Romain Laroche OffRL OnRL 35 10 0 02 Nov 2022
ERL-Re $^2$ : Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representation Jianye Hao Pengyi Li Hongyao Tang Yan Zheng Xian Fu Zhaopeng Meng 11 23 0 26 Oct 2022
DCE: Offline Reinforcement Learning With Double Conservative Estimates Chen Zhao K. Huang Chun yuan OffRL 22 1 0 27 Sep 2022
Sampling Through the Lens of Sequential Decision Making J. Dou Alvin Pan Runxue Bao Haiyi Mao Lei Luo Zhi-Hong Mao 24 19 0 17 Aug 2022
An Empirical Study of Implicit Regularization in Deep Offline RL Çağlar Gülçehre Srivatsan Srinivasan Jakub Sygnowski Georg Ostrovski Mehrdad Farajtabar Matt Hoffman Razvan Pascanu Arnaud Doucet OffRL 14 16 0 05 Jul 2022
Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination Jiafei Lyu Xiu Li Zongqing Lu OffRL 24 24 0 16 Jun 2022
Learning Dynamics and Generalization in Reinforcement Learning Clare Lyle Mark Rowland Will Dabney Marta Z. Kwiatkowska Y. Gal OOD OffRL 28 12 0 05 Jun 2022
Frustratingly Easy Regularization on Representation Can Boost Deep Reinforcement Learning Qiang He Huangyuan Su Jieyu Zhang Xinwen Hou OOD OffRL 20 6 0 29 May 2022
When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning? Aviral Kumar Joey Hong Anika Singh Sergey Levine OffRL 22 75 0 12 Apr 2022
A Workflow for Offline Model-Free Robotic Reinforcement Learning Aviral Kumar Anika Singh Stephen Tian Chelsea Finn Sergey Levine OffRL 138 84 0 22 Sep 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 138 279 0 12 Feb 2021
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 334 1,951 0 04 May 2020