Towards more precise coincidental correctness detection with Deep Semantic Learning
Publication Type
Journal Article
Publication Date
12-2024
Abstract
Coincidental correctness (CC) is a situation during the execution of a test case, the buggy entity is executed, but the program behaves correctly as expected. Many automated fault localization (FL) techniques use runtime information to discover the underlying connection between the executed buggy entity and the failing test result. The existence of CC will weaken such connection, mislead the FL algorithms to build inaccurate models, and consequently, decrease the localization accuracy. To alleviate the adverse effect of CC on FL, CC detection techniques have been proposed to identify the possible CC tests via heuristic or machine learning algorithms. However, their performance on precision is not satisfactory since they overestimate the possible CC tests and are insufficient in learning the deep semantic features. In this work, we propose a novel Triplet network-based Coincidental Correctness detection technique (i.e., TriCoCo) to overcome the limitations of the prior works. TriCoCo narrows the possible CC tests by designing three features to identify genuine passing tests. Instead of using all tests as inputs by existing techniques, TriCoCo takes the identified genuine passing tests and failing ones to train a triplet model that can evaluate their relative distance. Finally, TriCoCo infers the probability of being a CC test of the test in the rest of the passing tests by using the trained triplet model. We conduct large-scale experiments to evaluate TriCoCo based on the widely-used Defects4J benchmark. The results demonstrate that TriCoCo can improve not only the precision of CC detection but also the effectiveness of FL techniques, e.g., the precision of TriCoCo is 80.33% on average, and TriCoCo boosts the efficacy of DStar by 18% - 74% in terms of MFR metric when compared to seven state-of-the-art CC detection baselines.
Keywords
Coincidental correctness, Deep Semantic Learning, Fault localization, Feature extraction, Machine learning algorithms, Location awareness, Deep learning, Clustering algorithms, Prediction algorithms, Benchmark testing, Partitioning algorithms
Discipline
Artificial Intelligence and Robotics
Research Areas
Intelligent Systems and Optimization
Publication
IEEE Transactions on Software Engineering
Volume
50
Issue
12
First Page
3265
Last Page
3289
ISSN
0098-5589
Identifier
10.1109/TSE.2024.3481893
Publisher
Institute of Electrical and Electronics Engineers
Citation
XIE, Huan; LEI, Yan; YAN, Meng; LI, Shanshan; MAO, Xiaoguang; YU, Yue; and LO, David.
Towards more precise coincidental correctness detection with Deep Semantic Learning. (2024). IEEE Transactions on Software Engineering. 50, (12), 3265-3289.
Available at: https://ink.library.smu.edu.sg/sis_research/9920
Additional URL
https://doi.org/10.1109/TSE.2024.3481893
