Title

File-level defect prediction: Unsupervised vs. supervised models

Publication Type

Conference Proceeding Article

Publication Date

11-2017

Abstract

Background: Software defect models can help software quality assurance teams to allocate testing or code review resources. A variety of techniques have been used to build defect prediction models, including supervised and unsupervised methods. Recently, Yang et al. [1] surprisingly find that unsupervised models can perform statistically significantly better than supervised models in effort-aware change-level defect prediction. However, little is known about relative performance of unsupervised and supervised models for effort-aware file-level defect prediction. Goal: Inspired by their work, we aim to investigate whether a similar finding holds in effort-aware file-level defect prediction. Method: We replicate Yang et al.'s study on PROMISE dataset with totally ten projects. We compare the effectiveness of unsupervised and supervised prediction models for effort-aware file-level defect prediction. Results: We find that the conclusion of Yang et al. [1] does not hold under within-project but holds under cross-project setting for file-level defect prediction. In addition, following the recommendations given by the best unsupervised model, developers needs to inspect statistically significantly more files than that of supervised models considering the same inspection effort (i.e., LOC). Conclusions: (a) Unsupervised models do not perform statistically significantly better than state-of-art supervised model under within-project setting, (b) Unsupervised models can perform statistically significantly better than state-ofart supervised model under cross-project setting, (c) We suggest that not only LOC but also number of files needed to be inspected should be considered when evaluating effort-aware filelevel defect prediction models.

Discipline

Software Engineering | Theory and Algorithms

Research Areas

Intelligent Systems and Decision Analytics

Publication

Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017)

Identifier

10.1109/ESEM.2017.48

Publisher

IEEE

City or Country

Toronto, Canada

Additional URL

http://doi.org/10.1109/ESEM.2017.48

This document is currently not available here.

Share

COinS