Gray-Box Extraction of Execution Graphs for Anomaly Detection
Conference Proceeding Article
Many host-based anomaly detection systems monitor a process by observing the system calls it makes, and comparing these calls to a model of behavior for the program that the process should be executing. In this paper we introduce a new model of system call behavior, called an execution graph. The execution graph is the first such model that both requires no static analysis of the program source or binary, and conforms to the control flow graph of the program. When used as the model in an anomaly detection system monitoring system calls, it offers two strong properties: (i) it accepts only system call sequences that are consistent with the control flow graph of the program; (ii) it is maximal given a set of training data, meaning that any extensions to the execution graph could permit some intrusions to go undetected. In this paper, we formalize and prove these claims. We additionally evaluate the performance of our anomaly detection technique.
Information Security and Trust
CCS 2004: Proceedings of the 11th ACM Conference on Computer and Communications Security, October 25-29, 2004, Washington, DC
City or Country
Washington, DC, USA
GAO, Debin; Reiter, Michael K.; and SONG, Dawn.
Gray-Box Extraction of Execution Graphs for Anomaly Detection. (2004). CCS 2004: Proceedings of the 11th ACM Conference on Computer and Communications Security, October 25-29, 2004, Washington, DC. 318-329. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/1242