Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures

4 December 2018

Krishnamurthy Dvijotham

Papers citing "Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures"

46 / 46 papers shown

Title
The Pursuit of Diversity: Multi-Objective Testing of Deep Reinforcement Learning Agents Antony Bartlett Cynthia C. S. Liem Annibale Panichella 76 0 0 16 Oct 2025
Beyond expected value: geometric mean optimization for long-term policy performance in reinforcement learning Xinyi Sheng Dominik Baumann 116 0 0 29 Aug 2025
DiCriTest: Testing Scenario Generation for Decision-Making Agents Considering Diversity and Criticality Qitong Chu Yufeng Yue Danya Yao Huaxin Pei 85 0 0 15 Aug 2025
NPO: Learning Alignment and Meta-Alignment through Structured Human Feedback Madhava Gaikwad Ashwini Ramchandra Doke 185 0 0 22 Jul 2025
Towards Robust Deep Reinforcement Learning against Environmental State Perturbation Chenxu Wang Huaping Liu AAML 151 0 0 10 Jun 2025
Forecasting Rare Language Model Behaviors Erik Jones Meg Tong Jesse Mu Mohammed Mahfoud Jan Leike Roger C. Grosse Jared Kaplan William Fithian Ethan Perez Mrinank Sharma 271 4 0 24 Feb 2025
Test Where Decisions Matter: Importance-driven Testing for Deep Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2024 Stefan Pranger Hana Chockler Martin Tappler Bettina Könighofer OffRL 262 3 0 12 Nov 2024
Formal Verification and Control with Conformal Prediction Lars Lindemann Yiqi Zhao Xinyi Yu George J. Pappas Jyotirmoy Deshmukh 1.1K 33 0 31 Aug 2024
PTDRL: Parameter Tuning using Deep Reinforcement LearningIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2023 Elias Goldsztejn Tal Feiner Ronen I. Brafman 130 2 0 19 Jun 2023
Testing of Deep Reinforcement Learning Agents with Surrogate ModelsACM Transactions on Software Engineering and Methodology (TOSEM), 2023 Matteo Biagiola Paolo Tonella 215 30 0 22 May 2023
DIRECT: Learning from Sparse and Shifting Rewards using Discriminative Reward Co-Training Philipp Altmann Thomy Phan Fabian Ritz Thomas Gabor Claudia Linnhoff-Popien OffRL 118 1 0 18 Jan 2023
SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning Maxwell Standen Junae Kim Claudia Szabo AAML 208 6 0 11 Jan 2023
Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot CurriculaConference on Robot Learning (CoRL), 2022 Eli Bronstein S. Srinivasan Supratik Paul Aman Sinha Matthew O'Kelly Payam Nikdel Shimon Whiteson OffRL 227 19 0 02 Dec 2022
A Deep Reinforcement Learning Approach to Rare Event Estimation Anthony Corso Kyu-Young Kim Shubh Gupta Grace Gao Mykel J. Kochenderfer 104 0 0 22 Nov 2022
Data efficient surrogate modeling for engineering design: Ensemble-free batch mode deep active learning for regression Sarthak Kapoor Harsh Vardhan U. Timalsina Sumit Kumar Péter Völgyesi J. Sztipanovits AI4CE 142 12 0 16 Nov 2022
An Active Learning Reliability Method for Systems with Partially Defined Performance Functions Jonathan Sadeghi Romain Mueller John Redford 163 2 0 05 Oct 2022
Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error ModelsIEEE International Conference on Robotics and Automation (ICRA), 2022 Craig Innes S. Ramamoorthy AAML 143 16 0 20 Sep 2022
Rare event failure test case generation in Learning-Enabled-ControllersInternational Conference on Machine Learning Technologies (ICMLT), 2021 H. Vardhan J. Sztipanovits 140 20 0 11 Jun 2022
Exploring ML testing in practice -- Lessons learned from an interactive rapid review with Axis Communications Qunying Song Markus Borg Emelie Engström H. Ardö Sergio Rico 119 10 0 30 Mar 2022
Assisted Robust Reward Design Jerry Zhi-Yang He Anca Dragan 132 10 0 18 Nov 2021
Certifiable Deep Importance Sampling for Rare-Event Simulation of Black-Box Systems Mansur Arief Yuanlu Bai Wenhao Ding Shengyi He Zhiyuan Huang Henry Lam Ding Zhao 135 15 0 03 Nov 2021
Play to Grade: Testing Coding Games as Classifying Markov Decision Process Allen Nie Emma Brunskill Chris Piech 189 11 0 27 Oct 2021
A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation Jonathan Sadeghi Blaine Rogers James Gunn Thomas Saunders Sina Samangooei P. Dokania John Redford 260 8 0 28 Sep 2021
Learning to Give Checkable Answers with Prover-Verifier Games Cem Anil Guodong Zhang Yuhuai Wu Roger C. Grosse 194 19 0 27 Aug 2021
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature ReviewInternational Conference on Automated Software Engineering (ASE), 2021 Florian Tambon Gabriel Laberge Le An Amin Nikanjam Paulina Stevia Nouwou Mindom Y. Pequignot Foutse Khomh G. Antoniol E. Merlo François Laviolette 476 79 0 26 Jul 2021
Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event SamplingIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2021 Mengdi Xu Peide Huang Fengpei Li Jiacheng Zhu Xuewei Qi K. Oguchi Zhiyuan Huang Henry Lam Ding Zhao 152 4 0 19 Jun 2021
Adversarial Reinforcement Learning in Dynamic Channel Access and Power ControlIEEE Wireless Communications and Networking Conference (WCNC), 2021 Feng Wang M. C. Gursoy Senem Velipasalar AAML 106 14 0 12 May 2021
Understanding and Avoiding AI Failures: A Practical Guide R. M. Williams Roman V. Yampolskiy 179 28 0 22 Apr 2021
Causal Analysis of Agent Behavior for AI Safety Grégoire Delétang Jordi Grau-Moya Miljan Martic Tim Genewein Tom McGrath Vladimir Mikulik M. Kunesch Shane Legg Pedro A. Ortega CML 165 7 0 05 Mar 2021
Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance SamplersOperational Research (OR), 2021 Anand Deo Karthyek Murthy 232 12 0 14 Feb 2021
Evaluating the Robustness of Collaborative AgentsAdaptive Agents and Multi-Agent Systems (AAMAS), 2021 P. Knott Micah Carroll Sam Devlin K. Ciosek Katja Hofmann Anca Dragan Rohin Shah 150 41 0 14 Jan 2021
CoachNet: An Adversarial Sampling Approach for Reinforcement Learning E. Abolfathi Jun Luo Peyman Yadmellat K. Rezaee OnRL 141 3 0 07 Jan 2021
Transfer Learning for Efficient Iterative Safety ValidationAAAI Conference on Artificial Intelligence (AAAI), 2020 Anthony Corso Mykel J. Kochenderfer 132 5 0 09 Dec 2020
Rare-Event Simulation for Neural Network and Random Forest PredictorsACM Transactions on Modeling and Computer Simulation (TOMACS), 2020 Yuanlu Bai Zhiyuan Huang Henry Lam Ding Zhao 148 25 0 10 Oct 2020
Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems Aman Sinha Matthew O'Kelly Russ Tedrake John C. Duchi 297 53 0 24 Aug 2020
Deep Probabilistic Accelerated Evaluation: A Robust Certifiable Rare-Event Simulation Methodology for Black-Box Safety-Critical Systems Mansur Arief Zhiyuan Huang Guru Koushik Senthil Kumar Yuanlu Bai Shengyi He Wenhao Ding Henry Lam Ding Zhao 206 11 0 28 Jun 2020
A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical SystemsJournal of Artificial Intelligence Research (JAIR), 2020 Anthony Corso Robert J. Moss Mark Koren Ritchie Lee Mykel J. Kochenderfer 235 193 0 06 May 2020
Interpretable Safety Validation for Autonomous Vehicles Anthony Corso Mykel J. Kochenderfer 155 29 0 14 Apr 2020
Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition Anthony Corso Ritchie Lee Mykel J. Kochenderfer 152 24 0 14 Apr 2020
Robust Deep Reinforcement Learning against Adversarial Perturbations on State ObservationsNeural Information Processing Systems (NeurIPS), 2020 Huan Zhang Hongge Chen Chaowei Xiao Yue Liu Mingyan D. Liu Duane S. Boning Cho-Jui Hsieh AAML 381 339 0 19 Mar 2020
Analysing Deep Reinforcement Learning Agents Trained with Domain Randomisation Tianhong Dai Kai Arulkumaran Tamara Gerbert Samyakh Tukra Feryal M. P. Behbahani Anil Anthony Bharath 198 31 0 18 Dec 2019
Machine Learning Testing: Survey, Landscapes and HorizonsIEEE Transactions on Software Engineering (TSE), 2019 Jie M. Zhang Mark Harman Lei Ma Yang Liu VLM AILaw 236 813 0 19 Jun 2019
Predicting Model Failure using Saliency Maps in Autonomous Driving Systems Sina Mohseni Akshay V. Jagadeesh Zinan Lin 159 14 0 19 May 2019
Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents Christian Rupprecht Cyril Ibrahim C. Pal 143 34 0 02 Apr 2019
Scalable agent alignment via reward modeling: a research direction Jan Leike David M. Krueger Tom Everitt Miljan Martic Vishal Maini Shane Legg 332 514 0 19 Nov 2018
A Statistical Approach to Assessing Neural Network RobustnessInternational Conference on Learning Representations (ICLR), 2018 Stefan Webb Tom Rainforth Yee Whye Teh M. P. Kumar AAML 259 92 0 17 Nov 2018