GameRTS: A regression testing framework for video games

Publication Type

Conference Proceeding Article

Publication Date

5-2023

Abstract

Continuous game quality assurance is of great importance to satisfy the increasing demands of users. To respond to game issues reported by users timely, game com-panies often create and maintain a large number of releases, updates, and tweaks in a short time. Regression testing is an essential technique adopted to detect regression issues during the evolution of the game software. However, due to the special characteristics of game software (e.g., frequent updates and long-running tests), traditional regression testing techniques are not directly applicable. To bridge this gap, in this paper, we perform an early exploratory study to investigate the challenges in regression testing of video games. We first performed empirical studies to better understand the game development process, bugs introduced during game evolution, and the context sensitivity. Based on the results of the study, we proposed the first regression test selection (RTS) technique for game software, which is a compromise between safety and practicality. In particular, we model the test suite of game software as a State Transition Graph (STG) and then perform the RTS on the STG. We establish the dependencies between the states/actions of STG and game files, including game art resources, game design files, and source code, and perform change impact analysis to identify the states/actions (in the STG) that potentially execute such changes. We implemented our framework in a tool, named GameRTS, and evaluated its usefulness on 10 tasks of a large-scale commercial game, including a total of 1,429 commits over three versions. The experimental results demonstrate the usefulness and effectiveness of GameRTS in game RTS. For most tasks, GameRTS only selected one trace from STG, which can significantly reduce the testing time. Furthermore, GameRTS detects all the regression bugs from the test evaluation suites. Compared with the file-level RTS, GameRTS selected fewer states/actions/traces (i.e., 13.77%, 23.97%, 6.85%). In addition, GameRTS identified 2 new critical regression bugs in the game.

Keywords

Game testing, Regression testing, State transition graph, Testing cases selection

Discipline

Software Engineering | Theory and Algorithms

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of the 45th International Conference on Software Engineering, Melbourne, Australia, May 14-20

First Page

1393

Last Page

1404

Identifier

10.1109/ICSE48619.2023.00122

City or Country

Manhattan, New York City, US

Additional URL

https://doi.org/10.1109/ICSE48619.2023.00122

This document is currently not available here.

Share

COinS