Guiding Deep Learning System Testing using Surprise Adequacy

25 August 2018

Papers citing "Guiding Deep Learning System Testing using Surprise Adequacy"

50 / 66 papers shown

Title
FCGHunter: Towards Evaluating Robustness of Graph-Based Android Malware Detection Shiwen Song Xiaofei Xie Ruitao Feng Qi Guo Sen Chen AAML 45 0 0 28 Apr 2025
Towards Assessing Deep Learning Test Input Generators Seif Mzoughi Ahmed Hajyahmed Mohamed Elshafei Foutse Khomh anb Diego Elias Costa D. Costa AAML 40 0 0 03 Apr 2025
MetaSel: A Test Selection Approach for Fine-tuned DNN Models Amin Abbasishahkoo Mahboubeh Dadkhah Lionel C. Briand Dayi Lin 49 0 0 21 Mar 2025
Towards Comparable Active Learning Thorben Werner Johannes Burchert Lars Schmidt-Thieme 81 0 0 24 Feb 2025
Democratic Training Against Universal Adversarial Perturbations Bing-Jie Sun Jun Sun Wei Zhao AAML 66 0 0 08 Feb 2025
A3Rank: Augmentation Alignment Analysis for Prioritizing Overconfident Failing Samples for Deep Learning Models Zhengyuan Wei Haipeng Wang Qili Zhou William Chan 34 0 0 19 Jul 2024
Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification Ruben Grewal Paolo Tonella Andrea Stocco 48 12 0 29 Apr 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 53 6 0 12 Apr 2024
A Survey of Neural Network Robustness Assessment in Image Recognition Jie Wang Jun Ai Minyan Lu Haoran Su Dan Yu Yutao Zhang Junda Zhu Jingyu Liu AAML 30 3 0 12 Apr 2024
Machine Translation Testing via Syntactic Tree Pruning Quanjun Zhang Juan Zhai Chunrong Fang Jiawei Liu Weisong Sun Haichuan Hu Qingyu Wang 31 3 0 01 Jan 2024
Mutation-based Fault Localization of Deep Neural Networks Ali Ghanbari Deepak-George Thomas Muhammad Arbab Arshad Hridesh Rajan 13 12 0 10 Sep 2023
How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses Joana Cabral Costa Tiago Roxo Hugo Manuel Proença Pedro R. M. Inácio AAML 52 50 0 18 May 2023
DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks Zohreh Aghababaeyan Manel Abdellatif Mahboubeh Dadkhah Lionel C. Briand AAML 34 16 0 08 Mar 2023
Understanding the Complexity and Its Impact on Testing in ML-Enabled Systems Junming Cao Bihuan Chen Longjie Hu Jie Ying Gao Kaifeng Huang Xin Peng 26 3 0 10 Jan 2023
AmbieGen: A Search-based Framework for Autonomous Systems Testing D. Humeniuk Foutse Khomh G. Antoniol 27 13 0 01 Jan 2023
When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study Vincenzo Riccio Paolo Tonella AAML 24 29 0 21 Dec 2022
Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines Michael Weiss Paolo Tonella BDL UQCV 30 11 0 14 Dec 2022
An Empirical Study of Library Usage and Dependency in Deep Learning Frameworks Mohamed Raed El aoun L. Tidjon Ben Rombaut Foutse Khomh Ahmed E. Hassan 27 5 0 28 Nov 2022
An Overview of Structural Coverage Metrics for Testing Neural Networks Muhammad Usman Youcheng Sun D. Gopinath R. Dange Luca Manolache C. Păsăreanu 19 8 0 05 Aug 2022
Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing Michael Weiss A. Gómez Paolo Tonella AAML 18 6 0 21 Jul 2022
Guiding the retraining of convolutional neural networks against adversarial inputs Francisco Durán Silverio Martínez-Fernández Michael Felderer Xavier Franch AAML 38 1 0 08 Jul 2022
CodeS: Towards Code Model Generalization Under Distribution Shift Qiang Hu Yuejun Guo Xiaofei Xie Maxime Cordy Lei Ma Mike Papadakis Yves Le Traon OOD 39 10 0 11 Jun 2022
AEON: A Method for Automatic Evaluation of NLP Test Cases Jen-tse Huang Jianping Zhang Wenxuan Wang Pinjia He Yuxin Su Michael R. Lyu 40 23 0 13 May 2022
Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study) Michael Weiss Paolo Tonella AAML 18 50 0 02 May 2022
Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning Jun Guo Yonghong Chen Yihang Hao Zixin Yin Yin Yu Simin Li AAML 32 32 0 17 Apr 2022
Characterizing and Understanding the Behavior of Quantized Models for Reliable Deployment Qiang Hu Yuejun Guo Maxime Cordy Xiaofei Xie Wei Ma Mike Papadakis Yves Le Traon MQ 44 1 0 08 Apr 2022
Exploring ML testing in practice -- Lessons learned from an interactive rapid review with Axis Communications Qunying Song Markus Borg Emelie Engström H. Ardö Sergio Rico 14 10 0 30 Mar 2022
NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks Xiaofei Xie Tianlin Li Jian-Xun Wang Lei Ma Qing Guo Felix Juefei Xu Yang Liu AAML 21 51 0 24 Mar 2022
Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques M. K. Ahuja A. Gotlieb Helge Spieker AAML 19 4 0 24 Feb 2022
Security for Machine Learning-based Software Systems: a survey of threats, practices and challenges Huaming Chen Muhammad Ali Babar AAML 42 21 0 12 Jan 2022
Black-Box Testing of Deep Neural Networks Through Test Case Diversity Zohreh Aghababaeyan Manel Abdellatif Lionel C. Briand Ramesh S M. Bagherzadeh AAML 45 44 0 20 Dec 2021
Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterion Yuanyuan Yuan Qi Pang Shuai Wang 43 22 0 03 Dec 2021
Understanding Performance Problems in Deep Learning Systems Junming Cao Bihuan Chen Chao Sun Longjie Hu Shuai Wu Xin Peng 30 27 0 03 Dec 2021
ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks Hua Qi Zhijie Wang Qing Guo Jianlang Chen Felix Juefei Xu Lei Ma Jianjun Zhao AAML AI4CE 35 16 0 26 Nov 2021
Data Synthesis for Testing Black-Box Machine Learning Models Diptikalyan Saha Aniya Aggarwal Sandeep Hans 22 4 0 03 Nov 2021
A Methodology to Identify Cognition Gaps in Visual Recognition Applications Based on Convolutional Neural Networks Hannes Vietz Tristan Rauch Andreas Löcklin N. Jazdi M. Weyrich 15 4 0 05 Oct 2021
ML4ML: Automated Invariance Testing for Machine Learning Models Zukang Liao Pengfei Zhang Min Chen VLM 21 3 0 27 Sep 2021
DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score Vincenzo Riccio Nargiz Humbatova Gunel Jahangirova Paolo Tonella 23 36 0 15 Sep 2021
Distribution Awareness for AI System Testing David Berend 24 8 0 06 May 2021
Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks Jens Henriksson C. Berger Markus Borg Lars Tornberg S. Sathyamoorthy Cristofer Englund OODD 20 17 0 29 Mar 2021
A Review and Refinement of Surprise Adequacy Michael Weiss Rwiddhi Chakraborty Paolo Tonella AAML AI4TS 19 16 0 10 Mar 2021
Test Automation with Grad-CAM Heatmaps -- A Future Pipe Segment in MLOps for Vision AI? Markus Borg Ronald Jabangwe Simon Åberg Arvid Ekblom Ludwig Hedlund August Lidfeldt 22 18 0 02 Mar 2021
NEUROSPF: A tool for the Symbolic Analysis of Neural Networks Muhammad Usman Yannic Noller C. Păsăreanu Youcheng Sun D. Gopinath 32 8 0 27 Feb 2021
Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification Michael Weiss Paolo Tonella UQCV 23 20 0 29 Dec 2020
A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges G. Giray 33 120 0 14 Dec 2020
Software engineering for artificial intelligence and machine learning software: A systematic literature review E. Nascimento Anh Nguyen-Duc Ingrid Sundbø T. Conte 18 40 0 07 Nov 2020
Astraea: Grammar-based Fairness Testing E. Soremekun Sakshi Udeshi Sudipta Chattopadhyay 26 27 0 06 Oct 2020
SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation Sungmin Kang R. Feldt S. Yoo AAML 26 32 0 19 May 2020
Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty Xiyue Zhang Xiaofei Xie Lei Ma Xiaoning Du Q. Hu Yang Liu Jianjun Zhao Meng Sun AAML 16 76 0 24 Apr 2020
Manifold-based Test Generation for Image Classifiers Taejoon Byun Abhishek Vijayakumar Sanjai Rayadurgam D. Cofer 18 9 0 15 Feb 2020