Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

5-2019

Abstract

Software testing is an integral part of software development process. Unfortunately, for many projects, bugs are prevalent despite testing effort, and testing continues to cost significant amount of time and resources. This brings forward the issue of test case quality and prompts us to investigate what make good test cases. To answer this important question, we interview 21 and survey 261 practitioners, who come from many small to large companies and open source projects distributed in 27 countries, to create and validate 29 hypotheses that describe characteristics of good test cases and testing practices. These characteristics span multiple dimensions including test case contents, size and complexity, coverage, maintainability, and bug detection. We present highly rated characteristics and rationales why practitioners agree or disagree with them, which in turn highlight best practices and trade-offs that need to be considered in the creation of test cases. Our findings also highlight open problems and opportunities for software engineering researchers to improve practitioner activities and address their pain points.

Discipline

Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

Proceedings of the 41st ACM/IEEE International Conference on Software Engineering (ICSE 2019), Montreal, Canada, 2019 May 25-31

First Page

61

Last Page

70

Identifier

10.1109/ICSE-SEIP.2019.00015

City or Country

Montreal, Canada

Additional URL

https://doi.org/10.1109/ICSE-SEIP.2019.00015

Download

Share

COinS