Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

5-2022

Abstract

Code embedding is a keystone in the application of machine learning on several Software Engineering (SE) tasks. To effectively support a plethora of SE tasks, the embedding needs to capture program syntax and semantics in a way that is generic. To this end, we propose the first self-supervised pre-training approach (called Graphcode2vec) which produces task-agnostic embedding of lexical and program dependence features. Graphcode2vec achieves this via a synergistic combination of code analysis and Graph Neural Networks. Graphcode2vec is generic, it allows pre-training, and it is applicable to several SE downstream tasks. We evaluate the effectiveness of Graphcode2vec on four (4) tasks (method name prediction, solution classification, mutation testing and overfitted patch classification), and compare it with four (4) similarly generic code embedding baselines (Code2Seq, Code2Vec, CodeBERT, Graph-CodeBERT) and seven (7) task-specific, learning-based methods. In particular, Graphcode2vec is more effective than both generic and task-specific learning-based baselines. It is also complementary and comparable to GraphCodeBERT (a larger and more complex model). We also demonstrate through a probing and ablation study that Graphcode2vec learns lexical and program dependence features and that self-supervised pre-training improves effectiveness.

Keywords

Code analysis, Code embedding, Code representation

Discipline

Programming Languages and Compilers | Software Engineering

Research Areas

Information Systems and Management; Intelligent Systems and Optimization

Publication

Proceedings of the 2022 Mining Software Repositories Conference, Pittsburgh, United States, May 23-24

First Page

524

Last Page

536

ISBN

9781450393034

Identifier

10.1145/3524842.3528456

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3524842.3528456

Download

Share

COinS