Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

9-2019

Abstract

Coverage-guided greybox fuzzing has become one of the most common techniques for finding software bugs. Coverage metric, which decides how a fuzzer selects new seeds, is an essential parameter of fuzzing and can significantly affect the results. While there are many existing works on the effectiveness of different coverage metrics on software testing, little is known about how different coverage metrics could actually affect the fuzzing results in practice. More importantly, it is unclear whether there exists one coverage metric that is superior to all the other metrics. In this paper, we report the first systematic study on the impact of different coverage metrics in fuzzing. To this end, we formally define and discuss the concept of sensitivity, which can be used to theoretically compare different coverage metrics. We then present several coverage metrics with their variants. We conduct a study on these metrics with the DARPA CGC dataset, the LAVA-M dataset, and a set of real-world applications (a total of 221 binaries). We find that because each fuzzing instance has limited resources (time and computation power), (1) each metric has its unique merit in terms of flipping certain types of branches (thus vulnerability finding) and (2) there is no grand slam coverage metric that defeats all the others. We also explore combining different coverage metrics through cross-seeding, and the result is very encouraging: this pure fuzzing based approach can crash at least the same numbers of binaries in the CGC dataset as a previous approach (Driller) that combines fuzzing and concolic execution. At the same time, our approach uses fewer computing resources

Keywords

Computation power, Computing resource, Concolic execution, Coverage metrics, Real-world, Software bug, Systematic study; Vulnerability finding

Discipline

Information Security

Publication

Proceedings of the 22nd International Symposium on Research on Attacks, Intrusions and Defenses, Beijing, China, Sep 23-25

ISBN

9781939133076

Publisher

USENIX Association

City or Country

California, USA

Copyright Owner and License

Authors

Download

Share

COinS