An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods

15 November 2022

Papers citing "An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods"

50 / 73 papers shown

Title
Enhancing PPO with Trajectory-Aware Hybrid Policies Qisai Liu Zhanhong Jiang Hsin-Jung Yang Mahsa Khosravi Joshua R. Waite S. Sarkar 40 0 0 21 Feb 2025
A learning-based approach to stochastic optimal control under reach-avoid constraint Tingting Ni Maryam Kamgarpour 70 0 0 21 Dec 2024
Last-Iterate Convergence of General Parameterized Policies in Constrained MDPs Washim Uddin Mondal Vaneet Aggarwal 31 1 0 21 Aug 2024
Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments Han Wang Sihong He Zhili Zhang Fei Miao James Anderson 39 3 0 29 May 2024
Fast Stochastic Policy Gradient: Negative Momentum for Reinforcement Learning Haobin Zhang Zhuang Yang 25 0 0 08 May 2024
Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization Youbang Sun Tao-Wen Liu P. R. Kumar Shahin Shahrampour 35 0 0 04 May 2024
Learning Optimal Deterministic Policies with Stochastic Policy Gradients Alessandro Montenegro Marco Mussi Alberto Maria Metelli Matteo Papini 33 2 0 03 May 2024
Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis Guangchen Lan Dong-Jun Han Abolfazl Hashemi Vaneet Aggarwal Christopher G. Brinton 122 15 0 09 Apr 2024
Order-Optimal Regret with Novel Policy Gradient Approaches in Infinite-Horizon Average Reward MDPs Swetha Ganesh Washim Uddin Mondal Vaneet Aggarwal 39 3 0 02 Apr 2024
Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles Bhrij Patel Wesley A. Suttle Alec Koppel Vaneet Aggarwal Brian M. Sadler Amrit Singh Bedi Dinesh Manocha 32 1 0 18 Mar 2024
Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries Swetha Ganesh Jiayu Chen Gugan Thoppe Vaneet Aggarwal FedML 47 1 0 15 Mar 2024
On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimization Ling Liang Haizhao Yang 9 0 0 23 Jan 2024
Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction Jie Feng Ke Wei Jinchi Chen 15 1 0 02 Jan 2024
PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping Nai-Chieh Huang Ping-Chun Hsieh Kuo-Hao Ho I-Chen Wu 16 8 0 19 Dec 2023
A safe exploration approach to constrained Markov decision processes Tingting Ni Maryam Kamgarpour 20 3 0 01 Dec 2023
Improved Sample Complexity Analysis of Natural Policy Gradient Algorithm with General Parameterization for Infinite Horizon Discounted Reward Markov Decision Processes Washim Uddin Mondal Vaneet Aggarwal 22 9 0 18 Oct 2023
When is Agnostic Reinforcement Learning Statistically Tractable? Zeyu Jia Gene Li Alexander Rakhlin Ayush Sekhari Nathan Srebro OffRL 12 5 0 09 Oct 2023
Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates Guangchen Lan Han Wang James Anderson Christopher G. Brinton Vaneet Aggarwal FedML 16 27 0 09 Oct 2023
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR) Guo-qing Jiang Jinlong Liu Zixiang Ding Lin Guo W. Lin AI4CE 9 1 0 24 Sep 2023
Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach Leonardo F. Toso Han Wang James Anderson 27 2 0 19 Sep 2023
Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes Qinbo Bai Washim Uddin Mondal Vaneet Aggarwal 16 9 0 05 Sep 2023
On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization Mudit Gaur Amrit Singh Bedi Di-di Wang Vaneet Aggarwal 25 3 0 18 Jun 2023
Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space Anas Barakat Ilyas Fatkhullin Niao He 13 11 0 02 Jun 2023
Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Time Xiang Ji Gen Li OffRL 10 7 0 24 May 2023
Deep Metric Tensor Regularized Policy Gradient Gang Chen Victoria Huang 13 0 0 18 May 2023
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators Yin-Huan Han Meisam Razaviyayn Renyuan Xu 16 5 0 15 Mar 2023
Optimal Convergence Rate for Exact Policy Mirror Descent in Discounted Markov Decision Processes Emmeran Johnson Ciara Pike-Burke Patrick Rebeschini 26 11 0 22 Feb 2023
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies Ilyas Fatkhullin Anas Barakat Anastasia Kireeva Niao He 19 37 0 03 Feb 2023
A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence Carlo Alfano Rui Yuan Patrick Rebeschini 54 15 0 30 Jan 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 11 1 0 28 Jan 2023
Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global State Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 25 8 0 13 Jan 2023
Coordinate Ascent for Off-Policy RL with Global Convergence Guarantees Hsin-En Su Yen-Ju Chen Ping-Chun Hsieh Xi Liu OffRL 11 0 0 10 Dec 2022
On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization Mudit Gaur Vaneet Aggarwal Mridul Agarwal MLT 31 1 0 14 Nov 2022
Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Games Yan Chen Taoying Li 11 3 0 14 Oct 2022
SoftTreeMax: Policy Gradient with Tree Search Gal Dalal Assaf Hallak Shie Mannor Gal Chechik 11 1 0 28 Sep 2022
A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems Sihong He Yue Wang Shuo Han Shaofeng Zou Fei Miao 17 11 0 17 Sep 2022
On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 18 5 0 07 Sep 2022
An Approximate Policy Iteration Viewpoint of Actor-Critic Algorithms Zaiwei Chen S. T. Maguluri 11 0 0 05 Aug 2022
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithm Qinbo Bai Amrit Singh Bedi Vaneet Aggarwal 11 20 0 12 Jun 2022
Finite-Time Analysis of Fully Decentralized Single-Timescale Actor-Critic Qijun Luo Xiao Li 8 1 0 12 Jun 2022
Policy Optimization for Markov Games: Unified Framework and Faster Convergence Runyu Zhang Qinghua Liu Haiquan Wang Caiming Xiong Na Li Yu Bai 8 26 0 06 Jun 2022
Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs Dongsheng Ding K. Zhang Jiali Duan Tamer Bacsar Mihailo R. Jovanović 13 19 0 06 Jun 2022
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function Saeed Masiha Saber Salehkaleybar Niao He Negar Kiyavash Patrick Thiran 79 18 0 25 May 2022
Momentum-Based Policy Gradient with Second-Order Information Saber Salehkaleybar Sadegh Khorasani Negar Kiyavash Niao He Patrick Thiran 13 9 0 17 May 2022
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization Shicong Cen Fan Chen Yuejie Chi 16 15 0 12 Apr 2022
Deep Reinforcement Learning for Data-Driven Adaptive Scanning in Ptychography M. Schloz Johannes Müller T. Pekin W. V. D. Broek C. Koch 12 7 0 29 Mar 2022
Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction? Washim Uddin Mondal Vaneet Aggarwal S. Ukkusuri 15 9 0 28 Feb 2022
Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization Runlong Zhou Zelin He Yuandong Tian Yi Wu S. Du OffRL 12 3 0 11 Feb 2022
Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees Mo Zhou Jianfeng Lu 11 13 0 31 Jan 2022
Recent Advances in Reinforcement Learning in Finance B. Hambly Renyuan Xu Huining Yang OffRL 16 165 0 08 Dec 2021