Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
9-2023
Abstract
The reliability of decision-making policies is urgently important today as they have established the fundamentals of many critical applications, such as autonomous driving and robotics. To ensure reliability, there have been a number of research efforts on testing decision-making policies that solve Markov decision processes (MDPs). However, due to the deep neural network (DNN)-based inherit and infinite state space, developing scalable and effective testing frameworks for decision-making policies still remains open and challenging.In this paper, we present an effective testing framework for decision-making policies. The framework adopts a generative diffusion model-based test case generator that can easily adapt to different search spaces, ensuring the practicality and validity of test cases. Then, we propose a termination state novelty-based guidance to diversify agent behaviors and improve the test effectiveness. Finally, we evaluate the framework on five widely used benchmarks, including autonomous driving, aircraft collision avoidance, and gaming scenarios. The results demonstrate that our approach identifies more diverse and influential failure-triggering test cases compared to current state-of-the-art techniques. Moreover, we employ the detected failure cases to repair the evaluated models, achieving better robustness enhancement compared to the baseline method.
Keywords
generative model, testing, decision-making policies
Discipline
Databases and Information Systems | Software Engineering
Research Areas
Information Systems and Management
Publication
2023 38th IEEE/ACM International Conference on Automated Software Engineering: Luxembourg, September 11-15: Proceedings
First Page
243
Last Page
254
ISBN
9798350329964
Identifier
10.1109/ASE56229.2023.00153
Publisher
IEEE
City or Country
Piscataway, NJ
Citation
LI, Zhuo; WU, Xiongfei; ZHU, Derui; CHENG, Mingfei; CHEN, Siyuan; ZHANG, Fuyuan; XIE, Xiaofei; MA, Lei; and ZHAO, Jianjun.
Generative model-based testing on decision-making policies. (2023). 2023 38th IEEE/ACM International Conference on Automated Software Engineering: Luxembourg, September 11-15: Proceedings. 243-254.
Available at: https://ink.library.smu.edu.sg/sis_research/8270
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/ASE56229.2023.00153