The Gap Between Model-Based and Model-Free Methods on the Linear Quadratic Regulator: An Asymptotic Viewpoint

9 December 2018

Stephen Tu

Benjamin Recht

Papers citing "The Gap Between Model-Based and Model-Free Methods on the Linear Quadratic Regulator: An Asymptotic Viewpoint"

50 / 72 papers shown

Title
Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem Hesameddin Mohammadi Mohammad Tinati Stephen Tu Mahdi Soltanolkotabi M. Jovanović 78 0 0 24 Nov 2024
Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noise Ziyi Zhang Yorie Nakahira Guannan Qu 33 2 0 31 May 2024
On the Limited Representational Power of Value Functions and its Links to Statistical (In)Efficiency David Cheikhi Daniel Russo OffRL 58 0 0 11 Mar 2024
Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity Guhao Feng Han Zhong OffRL 76 2 0 28 Dec 2023
On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQR Jaeuk Shin Giho Kim Howon Lee Joonho Han Insoon Yang OffRL 41 1 0 09 Dec 2023
Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms Xiangyuan Zhang Weichao Mao S. Mowlavi M. Benosman Tamer Basar OffRL AI4CE 29 2 0 30 Nov 2023
On Representation Complexity of Model-based and Model-free Reinforcement Learning Hanlin Zhu Baihe Huang Stuart Russell OffRL 35 3 0 03 Oct 2023
Efficiency Separation between RL Methods: Model-Free, Model-Based and Goal-Conditioned Han Bao Raphaël Jungers Jean-Charles Delvenne OffRL 21 1 0 28 Sep 2023
Meta-Learning Operators to Optimality from Multi-Task Non-IID Data Thomas T. Zhang Leonardo F. Toso James Anderson Nikolai Matni 72 13 0 08 Aug 2023
Policy Gradient Methods for Discrete Time Linear Quadratic Regulator With Random Parameters Deyue Li 22 0 0 29 Mar 2023
The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithms Anirudh Vemula Yuda Song Aarti Singh J. Andrew Bagnell Sanjiban Choudhury OffRL 46 13 0 01 Mar 2023
Can Direct Latent Model Learning Solve Linear Quadratic Gaussian Control? Yi Tian Kaipeng Zhang Russ Tedrake S. Sra 47 4 0 30 Dec 2022
Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-off Zichen Zhang Johannes Kirschner Junxi Zhang Francesco Zanini Alex Ayoub Masood Dehghan Dale Schuurmans OffRL 24 3 0 17 Dec 2022
Near Sample-Optimal Reduction-based Policy Learning for Average Reward MDP Jinghan Wang Meng-Xian Wang Lin F. Yang 37 16 0 01 Dec 2022
$Learning Decentralized Linear Quadratic Regulators with $\sqrt{T}$ Regret$ Learning Decentralized Linear Quadratic Regulators with $\sqrt{T}$ Regret Lintao Ye Ming Chi Ruiquan Liao V. Gupta 16 1 0 17 Oct 2022
Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies Bin Hu Kaipeng Zhang Na Li M. Mesbahi Maryam Fazel Tamer Bacsar 89 27 0 10 Oct 2022
Statistical Learning Theory for Control: A Finite Sample Perspective Anastasios Tsiamis Ingvar M. Ziemann Nikolai Matni George J. Pappas 28 73 0 12 Sep 2022
Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic Regulator Xu-yang Chen Jingliang Duan Yingbin Liang Lin Zhao 32 6 0 18 Aug 2022
How are policy gradient methods affected by the limits of control? Ingvar M. Ziemann Anastasios Tsiamis H. Sandberg Nikolai Matni 25 14 0 14 Jun 2022
Rate-Optimal Online Convex Optimization in Adaptive Linear Control Asaf B. Cassel Alon Cohen Google Research 34 9 0 03 Jun 2022
Online No-regret Model-Based Meta RL for Personalized Navigation Yuda Song Ye Yuan Wen Sun Kris Kitani 44 0 0 05 Apr 2022
Learning Linear Models Using Distributed Iterative Hessian Sketching Han Wang James Anderson 21 2 0 08 Dec 2021
Generalized Decision Transformer for Offline Hindsight Information Matching Hiroki Furuta Y. Matsuo S. Gu OffRL 21 99 0 19 Nov 2021
On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structure Lintao Ye Haoqi Zhu V. Gupta 33 14 0 14 Oct 2021
Stabilizing Dynamical Systems via Policy Gradient Methods Juan C. Perdomo Jack Umenberger Max Simchowitz 40 44 0 13 Oct 2021
MBRL-Lib: A Modular Library for Model-based Reinforcement Learning Luis Pineda Brandon Amos Amy Zhang Nathan Lambert Roberto Calandra OffRL 33 46 0 20 Apr 2021
How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control? Jingxi Xu Bruce D. Lee Nikolai Matni Dinesh Jayaraman 105 6 0 02 Apr 2021
$Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with $\sqrt{T}$ Regret$ Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with $\sqrt{T}$ Regret Asaf B. Cassel Tomer Koren OffRL 36 17 0 25 Feb 2021
Using Echo State Networks to Approximate Value Functions for Control Allen G. Hart Kevin R. Olding Alexander M. G. Cox Olga Isupova Jonathan H.P Dawes 16 0 0 11 Feb 2021
Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity Kaipeng Zhang Xiangyuan Zhang Bin Hu Tamer Bacsar 21 19 0 04 Jan 2021
Policy Optimization for Markovian Jump Linear Quadratic Control: Gradient-Based Methods and Global Convergence Joao Paulo Jansch-Porto Bin Hu Geir Dullerud 27 8 0 24 Nov 2020
Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon B. Hambly Renyuan Xu Huining Yang 32 61 0 20 Nov 2020
Improved rates for prediction and identification of partially observed linear dynamical systems Holden Lee 25 10 0 19 Nov 2020
CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guarantee Tengyu Xu Yingbin Liang Guanghui Lan 52 122 0 11 Nov 2020
Safety-Critical Online Control with Adversarial Disturbances Bhaskar Ramasubramanian Baicen Xiao L. Bushnell Radha Poovendran AAML 18 1 0 20 Sep 2020
Certainty Equivalent Perception-Based Control Sarah Dean Benjamin Recht 21 28 0 27 Aug 2020
Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation Bo Pang Zhong-Ping Jiang 40 34 0 25 Aug 2020
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy Zuyue Fu Zhuoran Yang Zhaoran Wang 21 42 0 02 Aug 2020
Provably Efficient Model-based Policy Adaptation Yuda Song Aditi Mavalankar Wen Sun Sicun Gao TTA OffRL 22 9 0 14 Jun 2020
Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approach Guannan Qu Chenkai Yu S. Low Adam Wierman 22 19 0 12 Jun 2020
Policy Learning of MDPs with Mixed Continuous/Discrete Variables: A Case Study on Model-Free Control of Markovian Jump Systems Joao Paulo Jansch-Porto Bin Hu Geir Dullerud 17 16 0 04 Jun 2020
Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning Anoopkumar Sonar Vincent Pacelli Anirudha Majumdar 18 53 0 01 Jun 2020
On Regularizability and its Application to Online Control of Unstable LTI Systems S. Talebi Siavash Alemzadeh Niyousha Rahimi M. Mesbahi OffRL 8 12 0 29 May 2020
Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model Gen Li Yuting Wei Yuejie Chi Yuxin Chen 39 125 0 26 May 2020
Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms Tengyu Xu Zhe Wang Yingbin Liang 26 57 0 07 May 2020
Improving Sample Complexity Bounds for (Natural) Actor-Critic Algorithms Tengyu Xu Zhe Wang Yingbin Liang 27 25 0 27 Apr 2020
Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling Huaqing Xiong Tengyu Xu Yingbin Liang Wei Zhang 25 33 0 15 Feb 2020
Convergence Guarantees of Policy Optimization Methods for Markovian Jump Linear Systems Joao Paulo Jansch-Porto Bin Hu Geir Dullerud 25 35 0 10 Feb 2020
Natural Actor-Critic Converges Globally for Hierarchical Linear Quadratic Regulator Yuwei Luo Zhuoran Yang Zhaoran Wang Mladen Kolar 26 9 0 14 Dec 2019
Observational Overfitting in Reinforcement Learning Xingyou Song Yiding Jiang Stephen Tu Yilun Du Behnam Neyshabur OffRL 33 138 0 06 Dec 2019