Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
5-2023
Abstract
Previous studies have demonstrated that neural code comprehension models are vulnerable to identifier naming. By renaming as few as one identifier in the source code, the models would output completely irrelevant results, indicating that identifiers can be misleading for model prediction. However, identifiers are not completely detrimental to code comprehension, since the semantics of identifier names can be related to the program semantics. Well exploiting the two opposite impacts of identifiers is essential for enhancing the robustness and accuracy of neural code comprehension, and still remains under-explored. In this work, we propose to model the impact of identifiers from a novel causal perspective, and propose a counterfactual reasoning-based framework named CREAM. CREAM explicitly captures the misleading information of identifiers through multi-task learning in the training stage, and reduces the misleading impact by counterfactual inference in the inference stage. We evaluate CREAM on three popular neural code comprehension tasks, including function naming, defect detection and code classification. Experiment results show that CREAM not only significantly outperforms baselines in terms of robustness (e.g., +37.9% on the function naming task at F1 score), but also achieve improved results on the original datasets (e.g., +0.5% on the function naming task at F1 score).
Keywords
Code comprehension, Comprehension models, Counterfactuals, F1 scores, Misleading informations, Model prediction, Multitask learning, Neural code, Program semantics, Source codes
Discipline
Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
Proceedings of the 45th IEEE/ACM International Conference on Software Engineering, Melbourne, Australia, 2023 May 15-16
First Page
1933
Last Page
1945
ISBN
9781665457019
Identifier
10.1109/ICSE48619.2023.00164
Publisher
IEEE
City or Country
New Jersey
Citation
GAO, Shuzheng; GAO, Cuiyun; WANG, Chaozheng; SUN, Jun; LO, David; and YU, Yue.
Two sides of the same coin: Exploiting the impact of identifiers in neural code comprehension. (2023). Proceedings of the 45th IEEE/ACM International Conference on Software Engineering, Melbourne, Australia, 2023 May 15-16. 1933-1945.
Available at: https://ink.library.smu.edu.sg/sis_research/9270
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/ICSE48619.2023.00164