Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

11-2019

Abstract

—Game testing has been long recognized as a notoriously challenging task, which mainly relies on manual playing and scripting based testing in game industry. Even until recently, automated game testing still remains to be largely untouched niche. A key challenge is that game testing often requires to play the game as a sequential decision process. A bug may only be triggered until completing certain difficult intermediate tasks, which requires a certain level of intelligence. The recent success of deep reinforcement learning (DRL) sheds light on advancing automated game testing, without human competitive intelligent support. However, the existing DRLs mostly focus on winning the game rather than game testing. To bridge the gap, in this paper, we first perform an in-depth analysis of 1349 real bugs from four real-world commercial game products. Based on this, we propose four oracles to support automated game testing, and further propose Wuji, an on-the-fly game testing framework, which leverages evolutionary algorithms, DRL and multi-objective optimization to perform automatic game testing. Wuji balances between winning the game and exploring the space of the game. Winning the game allows the agent to make progress in the game, while space exploration increases the possibility of discovering bugs. We conduct a large-scale evaluation on a simple game and two popular commercial games. The results demonstrate the effectiveness of Wuji in exploring space and detecting bugs. Moreover, Wuji found 3 previously unknown bugs1 , which have been confirmed by the developers, in the commercial games

Keywords

Testing, Artificial Intelligence, Deep Reinforcement Learning, Evolutionary Multi-Objective Optimizatio

Discipline

Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering, San Diego, 2019 November 11-15

First Page

1

Last Page

13

ISBN

9781728125091

Identifier

10.1109/ASE.2019.00077

Publisher

IEEE Press

City or Country

San Diego, California

Download

Share

COinS