Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
3-2022
Abstract
Statistical language models on source code have successfully assisted software engineering tasks. However, developers can create or pick arbitrary identifiers when writing source code. Freely chosen identifiers lead to the notorious out-of-vocabulary (OOV) problem that negatively affects model performance. Recently, Karampatsis et al. showed that using the Byte Pair Encoding (BPE) algorithm to address the OOV problem can improve the language models’ predictive performance on source code. However, a drawback of BPE is that it cannot split the identifiers in a way that preserves the meaningful semantics. Prior researchers also show that splitting compound identifiers into sub-words that reflect the semantics can benefit software development tools. These two facts motivate us to explore whether identifier splitting techniques can be utilized to augment the BPE algorithm and boost the performance of open-vocabulary language models considered in Karampatsis et al.’s work. This paper proposes to split identifiers in both constructing vocabulary and processing model inputs procedures, thus exploiting three different settings of applying identifier splitting to language models for the code completion task. We contrast models’ performance under these settings and find that simply inserting identifier splitting into the pipeline hurts the model performance, while a hybrid strategy combining identifier splitting and the BPE algorithm can outperform the original open-vocabulary models on predicting identifiers by 3.68% of recall and 6.32% of Mean Reciprocal Rank. The results also show that the hybrid strategy can improve the entropy of language models by 2.02%.
Keywords
Open vocabulary, Identifier splitting, Language model of code
Discipline
Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER): Honolulu, March 15-18: Proceedings
First Page
1134
Last Page
1138
ISBN
9781665437868
Identifier
10.1109/SANER53432.2022.00130
Publisher
IEEE
City or Country
Piscataway, NJ
Citation
SHI, Jieke; YANG, Zhou; HE, Junda; XU, Bowen; and LO, David.
Can identifier splitting improve open-vocabulary language model of code?. (2022). 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER): Honolulu, March 15-18: Proceedings. 1134-1138.
Available at: https://ink.library.smu.edu.sg/sis_research/7698
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/SANER53432.2022.00130