Publication Type
Journal Article
Version
publishedVersion
Publication Date
11-2023
Abstract
pplying deep learning (DL) to science is a new trend in recent years, which leads DL engineering to become an important problem. Although training data preparation, model architecture design, and model training are the normal processes to build DL models, all of them are complex and costly. Therefore, reusing the open-sourced pre-trained model is a practical way to bypass this hurdle for developers. Given a specific task, developers can collect massive pre-trained deep neural networks from public sources for reusing. However, testing the performance (e.g., accuracy and robustness) of multiple deep neural networks (DNNs) and recommending which model should be used is challenging regarding the scarcity of labeled data and the demand for domain expertise. In this article, we propose a labeling-free (LaF) model selection approach to overcome the limitations of labeling efforts for automated model reusing. The main idea is to statistically learn a Bayesian model to infer the models’ specialty only based on predicted labels. We evaluate LaF using nine benchmark datasets, including image, text, and source code, and 165 DNNs, considering both the accuracy and robustness of models. The experimental results demonstrate that LaF outperforms the baseline methods by up to 0.74 and 0.53 on Spearman’s correlation and Kendall’s τ, respectively.
Keywords
Computing methodologies, Artificial intelligence, Software and its engineering, Software creation and management, Software development process management
Discipline
Databases and Information Systems
Research Areas
Data Science and Engineering; Cybersecurity
Publication
ACM Transactions on Software Engineering and Methodology
Volume
33
Issue
1
First Page
1
Last Page
28
ISSN
1049-331X
Identifier
10.1145/3611666
Publisher
Association for Computing Machinery (ACM)
Citation
HU, Qiang; GUO, Yuejun; XIE, Xiaofei; CORDY, Maxime; PAPADAKIS, Mike; and TRAON, Yves Le.
LaF: Labeling-free model selection for automated deep neural network reusing. (2023). ACM Transactions on Software Engineering and Methodology. 33, (1), 1-28.
Available at: https://ink.library.smu.edu.sg/sis_research/8476
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3611666