Least-Squares Temporal Difference Learning for the Linear Quadratic Regulator

International Conference on Machine Learning (ICML), 2017

22 December 2017

Stephen Tu

Benjamin Recht

OffRL

ArXiv (abs)PDF HTML

Papers citing "Least-Squares Temporal Difference Learning for the Linear Quadratic Regulator"

50 / 69 papers shown

Title
The Confusing Instance Principle for Online Linear Quadratic Control Waris Radji Odalric-Ambrym Maillard OffRL 104 1 0 22 Oct 2025
Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic RegulatorInternational Joint Conference on Artificial Intelligence (IJCAI), 2024 Xuyang Chen Jingliang Duan Tianyuan Chen 250 1 0 02 May 2025
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 195 0 0 15 Apr 2025
Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem Hesameddin Mohammadi Mohammad Tinati Stephen Tu Mahdi Soltanolkotabi M. Jovanović 265 1 0 24 Nov 2024
Coordinating Planning and Tracking in Layered Control Policies via Actor-Critic LearningIEEE Conference on Decision and Control (CDC), 2024 Fengjun Yang Nikolai Matni OffRL 156 0 0 03 Aug 2024
Single Trajectory Conformal Prediction Brian Lee Nikolai Matni 397 2 0 03 Jun 2024
On the Limited Representational Power of Value Functions and its Links to Statistical (In)Efficiency David Cheikhi Daniel Russo OffRL 194 0 0 11 Mar 2024
Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Range Yuzi Yan Yuan-Chung Shen 164 1 0 05 Mar 2024
From Spectral Theorem to Statistical Independence with Application to System Identification Muhammad Naeem Amir Khazraei Miroslav Pajic 117 2 0 16 Oct 2023
Data-Driven H-infinity Control with a Real-Time and Efficient Reinforcement Learning Algorithm: An Application to Autonomous Mobility-on-Demand Systems Ali Aalipour Alireza Khani 79 6 0 16 Sep 2023
Meta-Learning Operators to Optimality from Multi-Task Non-IID DataInternational Conference on Learning Representations (ICLR), 2023 Thomas T. Zhang Leonardo F. Toso James Anderson Nikolai Matni 243 14 0 08 Aug 2023
The Optimal Approximation Factors in Misspecified Off-Policy Value Function EstimationInternational Conference on Machine Learning (ICML), 2023 Philip Amortila Nan Jiang Csaba Szepesvári OffRL 216 4 0 25 Jul 2023
Reinforcement Learning with Partial Parametric Model KnowledgeIFAC-PapersOnLine (IFAC-PapersOnLine), 2023 Shuyuan Wang Philip D. Loewen Nathan P. Lawrence M. Forbes R. Bhushan Gopaluni KELM 90 0 0 26 Apr 2023
Learning and Concentration for High Dimensional Linear Gaussians: an Invariant Subspace Approach Muhammad Naeem 125 2 0 04 Apr 2023
Policy Evaluation in Distributional LQRConference on Learning for Dynamics & Control (L4DC), 2023 Zifan Wang Yulong Gao Si Wang Michael M. Zavlanos Alessandro Abate Karl H. Johansson OffRL 166 4 0 23 Mar 2023
Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation Zhifa Ke Junyu Zhang Zaiwen Wen 154 0 0 25 Feb 2023
Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-offNeural Information Processing Systems (NeurIPS), 2022 Zichen Zhang Johannes Kirschner Junxi Zhang Francesco Zanini Alex Ayoub Masood Dehghan Dale Schuurmans OffRL 268 3 0 17 Dec 2022
Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic ApproachConference on Learning for Dynamics & Control (L4DC), 2022 Muhammad Naeem Miroslav Pajic 292 1 0 07 Dec 2022
Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic RegulatorAAAI Conference on Artificial Intelligence (AAAI), 2022 Xu-yang Chen Jingliang Duan Yingbin Liang Tianyuan Chen 174 9 0 18 Aug 2022
Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State SpaceConference on Learning for Dynamics & Control (L4DC), 2022 Muhammad Naeem Miroslav Pajic 160 3 0 25 May 2022
A Complete Characterization of Linear Estimators for Offline Policy EvaluationJournal of machine learning research (JMLR), 2022 Juan C. Perdomo A. Krishnamurthy Peter L. Bartlett Sham Kakade OffRL 232 4 0 08 Mar 2022
Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence GuaranteesJournal of machine learning research (JMLR), 2022 Mo Zhou Jianfeng Lu 259 18 0 31 Jan 2022
Augmented RBMLE-UCB Approach for Adaptive Control of Linear Quadratic SystemsNeural Information Processing Systems (NeurIPS), 2022 Akshay Mete Rahul Singh P. R. Kumar 89 9 0 25 Jan 2022
Convergence and Optimality of Policy Gradient Methods in Weakly Smooth SettingsAAAI Conference on Artificial Intelligence (AAAI), 2021 Matthew Shunshi Zhang Murat A. Erdogdu Animesh Garg 255 5 0 30 Oct 2021
Local policy search with Bayesian optimizationNeural Information Processing Systems (NeurIPS), 2021 Sarah Müller Alexander von Rohr Sebastian Trimpe BDL 196 52 0 22 Jun 2021
Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite HorizonSIAM Journal of Control and Optimization (SICON), 2020 B. Hambly Renyuan Xu Huining Yang 220 74 0 20 Nov 2020
System Identification via Meta-Learning in Linear Time-Varying Environments Sen Lin Hang Wang Junshan Zhang OffRL 199 3 0 27 Oct 2020
Robust Reinforcement Learning: A Case Study in Linear Quadratic RegulationAAAI Conference on Artificial Intelligence (AAAI), 2020 Bo Pang Zhong-Ping Jiang 284 38 0 25 Aug 2020
Reinforcement Learning with Fast Stabilization in Linear Dynamical SystemsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2020 Sahin Lale Kamyar Azizzadenesheli B. Hassibi Anima Anandkumar 172 39 0 23 Jul 2020
Structured Policy Iteration for Linear Quadratic RegulatorInternational Conference on Machine Learning (ICML), 2020 Youngsuk Park Ryan A. Rossi Zheng Wen Gang Wu Handong Zhao OffRL 100 22 0 13 Jul 2020
Learning Expected Reward for Switched Linear Control Systems: A Non-Asymptotic View Muhammad Naeem Miroslav Pajic 152 1 0 15 Jun 2020
Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approach Guannan Qu Chenkai Yu S. Low Adam Wierman 119 19 0 12 Jun 2020
Learning nonlinear dynamical systems from a single trajectoryConference on Learning for Dynamics & Control (L4DC), 2020 Dylan J. Foster Alexander Rakhlin Tuhin Sarkar 129 78 0 30 Apr 2020
Transfer Reinforcement Learning under Unobserved Contextual InformationInternational Conference on Cyber-Physical Systems (ICCPS), 2020 Yan Zhang Michael M. Zavlanos OffRL 122 7 0 09 Mar 2020
Convergence Guarantees of Policy Optimization Methods for Markovian Jump Linear SystemsAmerican Control Conference (ACC), 2020 Joao Paulo Jansch-Porto Bin Hu Geir Dullerud 140 37 0 10 Feb 2020
Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical SystemsConference on Uncertainty in Artificial Intelligence (UAI), 2020 S. Asghari Ouyang Yi A. Nayyar 191 11 0 27 Jan 2020
Naive Exploration is Optimal for Online LQRInternational Conference on Machine Learning (ICML), 2020 Max Simchowitz Dylan J. Foster 273 194 0 27 Jan 2020
Probabilistic Safety Constraints for Learned High Relative Degree System DynamicsConference on Learning for Dynamics & Control (L4DC), 2019 M. J. Khojasteh Vikas Dhiman M. Franceschetti Nikolay Atanasov 389 74 0 20 Dec 2019
Natural Actor-Critic Converges Globally for Hierarchical Linear Quadratic Regulator Yuwei Luo Zhuoran Yang Zhaoran Wang Mladen Kolar 232 10 0 14 Dec 2019
Augmented Random Search for Quadcopter Control: An alternative to Reinforcement LearningInternational Journal of Information Technology and Computer Science (IJITCS), 2019 A. K. Tiwari Sandeep Varma Nadimpalli 100 3 0 28 Nov 2019
Statistical Learning for Analysis of Networked Control Systems over Unknown Channels Konstantinos Gatsis George J. Pappas 73 11 0 08 Nov 2019
Continuous Control with Contexts, Provably S. Du Ruosong Wang Mengdi Wang Lin F. Yang OffRL 100 5 0 30 Oct 2019
Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field GamesInternational Conference on Learning Representations (ICLR), 2019 Zuyue Fu Zhuoran Yang Yongxin Chen Zhaoran Wang 194 57 0 16 Oct 2019
Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function ApproximationSIAM Journal on Mathematics of Data Science (SIMODS), 2019 Thinh T. Doan S. T. Maguluri Justin Romberg 201 44 0 25 Jul 2019
Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties A. Marco Dominik Baumann Philipp Hennig Sebastian Trimpe 104 3 0 24 Jul 2019
Alice's Adventures in the Markovian World Zhanzhan Zhao Haoran Sun 120 0 0 21 Jul 2019
On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost Zhuoran Yang Yongxin Chen Mingyi Hong Zhaoran Wang 240 40 0 14 Jul 2019
From self-tuning regulators to reinforcement learning and back againIEEE Conference on Decision and Control (CDC), 2019 Nikolai Matni Alexandre Proutiere Anders Rantzer Stephen Tu 243 90 0 27 Jun 2019
Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic RegulatorNeural Information Processing Systems (NeurIPS), 2019 K. Krauth Stephen Tu Benjamin Recht 167 67 0 30 May 2019
Learning robust control for LQR systems with multiplicative noise via policy gradient Benjamin J. Gravell Peyman Mohajerin Esfahani Tyler H. Summers 260 27 0 28 May 2019