Publication Type

Journal Article

Version

acceptedVersion

Publication Date

3-2026

Abstract

Perception systems are vital for the safety of autonomous driving. In complex autonomous driving scenarios, autonomous vehicles must overcome various natural hazards, such as heavy rain or raindrops on the camera lens. Therefore, it is essential to conduct comprehensive testing of the perception systems in autonomous vehicles against these hazards, as demanded by the regulatory agencies of many countries for human drivers. Since there are many hazard scenarios, each of which has multiple configurable parameters, the challenges are (1) how do we systematically and adequately test an autonomous vehicle against these hazard scenarios, with measurable outcome; and (2) how do we efficiently explore the huge search space to identify scenarios that would induce failure? In this work, we propose a Hazards Generation and Testing framework (HazGT) to generate a customizable and comprehensive repository of hazard scenarios for evaluating the perception system of autonomous vehicles. HazGT not only allows us to measure how comprehensively an autonomous vehicle (AV) has been tested against different hazards but also supports the identification of important hazards through optimization. HazGT supports a total of 70 kinds of hazards relevant to the visual perception of AVs, which are based on industrial regulations. HazGT automatically optimizes the parameter values to efficiently achieve different testing objectives. We have implemented HazGT based on two popular 3D engines, i.e., Unity and Unreal Engine. For the two mainstream perception models (i.e., YOLO and Faster RCNN), we have evaluated their performance against each hazard through extensive experiments, and the results show that both systems have much room to improve. In addition, our experiments also found that ChatGPT4 performs slightly worse than YOLO. Our optimization-based testing system is effective in finding perceptual errors in the perception models. The hazard images generated by HazGT are instrumental for improving perception models.

Keywords

Autonomous vehicle, Perception system, Testing, Traffic hazards generation

Discipline

Artificial Intelligence and Robotics | Numerical Analysis and Scientific Computing | Transportation

Research Areas

Data Science and Engineering

Publication

Information Fusion

Volume

127

First Page

1

Last Page

16

ISSN

1566-2535

Identifier

10.1016/j.inffus.2025.103729

Publisher

Elsevier

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1016/j.inffus.2025.103729

Download

Share

COinS