Publication Type

Conference Proceeding Article

Publication Date



Cyber security is increasingly important for defending computer systems from loss of privacy or unauthorised use. One important aspect is threat analysis - how does an attacker infiltrate a system and what do they want once they are inside. This paper considers the problem of Active Malware Analysis, where we learn about the human or software intruder by actively interacting with it with the goal of learning about its behaviours and intentions, whilst at the same time that intruder may be trying to avoid detection or showing those behaviours and intentions. This game-theoretic active learning is then used to obtain a behavioural clustering of malware, an important contribution for both understanding malware at a high level and more crucially, for the deployment of effective anti-malware defences. This paper makes the following contributions: (i) A formal definition of the game-theoretic active malware analysis problem; (ii) A fast algorithm for learning about a malware in the active analysis problem which utilises the concept of reducing entropy in the beliefs about the malware; (iii) A virtual machine based agent architecture for the implementation of the active malware analysis problem and (iv) A behaviour based clustering of malware behaviour which is shown to be more accurate than a similar clustering using only passive information about the malware.


Malware Analysis, Stochastic Game, Autonomous agents, Computer crime, Game theory, Intrusion detection, Multi agent systems, Network security


Artificial Intelligence and Robotics | Information Security

Research Areas

Intelligent Systems and Decision Analytics; Cybersecurity


Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012): Valencia, Spain, June 4-8

First Page


Last Page






City or Country

Richland, SC

Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Additional URL