Empirical evaluation of hyper-parameter optimization techniques for deep learning-based malware detectors
Publication Type
Conference Proceeding Article
Publication Date
9-2024
Abstract
In machine learning, hyper-parameter optimization (HPO) aims to tune the set of parameters that controls the learning process. HPO could be time-consuming and resource-intensive due to the huge parameter search space and the complexity of models such as deep neural networks. Many of the existing HPO techniques tend to be variants of Bayesian optimization methods; each of which has been applied successfully for model tuning in different application domains. However, these Bayesian optimization methods have not been systematically evaluated against each other in the context of deep learning based malware detection. In this paper, we report a large-scale empirical study comparing popular HPO techniques on the performance of deep learning based malware classifiers. We use a diverse collection of seven datasets covering the most typical features used in malware detection. We conduct our experiments with Ray Tune, a distributed tuning platform, and popular optimization libraries such as Optuna, HyperOpt, Nevergrad, etc., across a wide range of computing platforms including AWS EC2, high-performance workstation, and laptop computers. Our extensive experiments provide useful insights into the application of different HPO techniques in deep learning based malware detection.
Keywords
Malware detection, Hyper-parameter tuning, Bayesian optimization, Deep learning
Discipline
Information Security
Research Areas
Data Science and Engineering; Cybersecurity
Publication
Proceedings of the 28th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems : Seville, Spain, September 11-13
Volume
246
First Page
2090
Last Page
2099
Identifier
10.1016/j.procs.2024.09.640
Publisher
Elsevier ScienceDirect
City or Country
Spain
Citation
SHAR, Lwin Khin; TA, Nguyen Binh Duong; YEO, Yao Cong; and FAN, Jiani.
Empirical evaluation of hyper-parameter optimization techniques for deep learning-based malware detectors. (2024). Proceedings of the 28th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems : Seville, Spain, September 11-13. 246, 2090-2099.
Available at: https://ink.library.smu.edu.sg/sis_research/9797
Additional URL
https://doi.org/10.1016/j.procs.2024.09.640