Publication Type
Conference Proceeding Article
Publication Date
11-2017
Abstract
Background: Software defect models can help software quality assurance teams to allocate testing or code review resources. A variety of techniques have been used to build defect prediction models, including supervised and unsupervised methods. Recently, Yang et al. [1] surprisingly find that unsupervised models can perform statistically significantly better than supervised models in effort-aware change-level defect prediction. However, little is known about relative performance of unsupervised and supervised models for effort-aware file-level defect prediction. Goal: Inspired by their work, we aim to investigate whether a similar finding holds in effort-aware file-level defect prediction. Method: We replicate Yang et al.'s study on PROMISE dataset with totally ten projects. We compare the effectiveness of unsupervised and supervised prediction models for effort-aware file-level defect prediction. Results: We find that the conclusion of Yang et al. [1] does not hold under within-project but holds under cross-project setting for file-level defect prediction. In addition, following the recommendations given by the best unsupervised model, developers needs to inspect statistically significantly more files than that of supervised models considering the same inspection effort (i.e., LOC). Conclusions: (a) Unsupervised models do not perform statistically significantly better than state-of-art supervised model under within-project setting, (b) Unsupervised models can perform statistically significantly better than state-ofart supervised model under cross-project setting, (c) We suggest that not only LOC but also number of files needed to be inspected should be considered when evaluating effort-aware filelevel defect prediction models.
Discipline
Software Engineering | Theory and Algorithms
Publication
Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017)
Identifier
10.1109/ESEM.2017.48
Publisher
IEEE
City or Country
Toronto, Canada
Citation
YAN, Meng; FANG, Yicheng; LO, David; XIA, Xin; and ZHANG, Xiaohong.
File-level defect prediction: Unsupervised vs. supervised models. (2017). Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017).
Available at: https://ink.library.smu.edu.sg/sis_research/3923
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://doi.org/10.1109/ESEM.2017.48