Diversity-oriented testing for competitive game agent via constraint-guided adversarial agent training

Publication Type

Journal Article

Publication Date

1-2025

Abstract

Deep reinforcement learning has achieved remarkable success in competitive games, surpassing human performance in applications ranging from business competitions to video games. In competitive environments, agents face the challenge of adapting to continuously shifting adversary strategies, necessitating the ability to handle diverse scenarios. Existing studies primarily focus on evaluating agent robustness either through perturbing observations, which has practical limitations, or through training adversarial agents to expose weaknesses, which lacks strategy diversity exploration. There are also studies which rely on curiosity-based mechanism to explore the diversity, yet they may lack direct guidance to enhance identified decision-making flaws. In this paper, we propose a novel diversity-oriented testing framework (called AdvTest) to test the competitive game agent via constraint-guided adversarial agent training. Specifically, AdvTest adds constraints as the explicit guidance during adversarial agent training to make it capable of defeating the target agent using diverse strategies. To realize the method, three challenges need to be addressed, i.e., what are the suitable constraints, when to introduce constraints, and which constraint should be added. We experimentally evaluate AdvTest on the commonly-used competitive game environment, StarCraft II. The results on four maps show that AdvTest exposes more diverse failure scenarios compared with the commonly-used and state-of-the-art baselines.

Keywords

Training, Testing, Games, Reinforcement Learning, Decision Making, Business, Safety, Robustness, Fault Diagnosis, Diversity Methods, Adversarial Agent, Constrained Reinforcement Learning, Testing Diversity

Discipline

Software Engineering

Research Areas

Intelligent Systems and Optimization

Publication

IEEE Transactions on Software Engineering

Volume

51

Issue

1

First Page

66

Last Page

81

ISSN

0098-5589

Identifier

10.1109/TSE.2024.3491193

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TSE.2024.3491193

This document is currently not available here.

Share

COinS