Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

7-2022

Abstract

Hitherto statistical type inference systems rely thoroughly on supervised learning approaches, which require laborious manual effort to collect and label large amounts of data. Most Turing-complete imperative languages share similar control- and data-flow structures, which make it possible to transfer knowledge learned from one language to another. In this paper, we propose a cross-lingual transfer learning framework, Plato, for statistical type inference, which allows us to leverage prior knowledge learned from the labeled dataset of one language and transfer it to the others, e.g., Python to JavaScript, Java to JavaScript, etc. Plato is powered by a novel kernelized attention mechanism to constrain the attention scope of the backbone Transformer model such that model is forced to base its prediction on commonly shared features among languages. In addition, we propose the syntax enhancement that augments the learning on the feature overlap among language domains. Furthermore, Plato can also be used to improve the performance of the conventional supervised-based type inference by introducing crosslanguage augmentation, which enables the model to learn more general features across multiple languages. We evaluated Plato under two settings: 1) under the cross-domain scenario that the target language data is not labeled or labeled partially, the results show that Plato outperforms the state-of-the-art domain transfer techniques by a large margin, e.g., it improves the Python to TypeScript baseline by +14.6%@EM, +18.6%@weighted-F1, and 2) under the conventional monolingual supervised scenario, Plato improves the Python baseline by +4.10%@EM, +1.90%@weighted-F1 with the introduction of the cross-lingual augmentation.

Keywords

Deep Learning, Transfer Learning, Type Inference

Discipline

Databases and Information Systems | Software Engineering

Research Areas

Information Systems and Management

Publication

Proceedings of the 31th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Conference, 2022 July 18-22

First Page

239

Last Page

250

ISBN

9781450393799

Identifier

10.1145/3533767.3534411

Publisher

ACM

City or Country

Virtual Conference

Additional URL

https://doi.org/10.1145/3533767.3534411

Download

Share

COinS