Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards

Publication Type

Conference Proceeding Article

Publication Date

12-2019

Abstract

In the complex multi-agent tasks, various agents must cooperate to distribute relevant subtasks among each other to achieve joint task objectives. An agent's choice of the relevant subtask changes over time with the changes in the task environment state. Multi-agent Hierarchical Reinforcement Learning (MAHRL) provides an approach for learning to select the subtasks in response to the environment states, by using the joint task rewards to train various agents. When the joint task involves complex inter-agent dependencies, only a subset of agents might be capable of reaching the rewarding task states while other agents take precursory or intermediate roles. The delayed task reward might not be sufficient in such tasks to learn the coordinating policies for various agents. In this paper, we introduce a novel approach of MAHRL called Inter-Subtask Empowerment based Multi-agent Options (ISEMO) in which an Inter-Subtask Empowerment Reward (ISER) is given to an agent which enables the precondition(s) of other agents' subtasks. ISER is given in addition to the domain task reward in order to improve the inter-agent coordination. ISEMO also incorporates options model that can learn parameterized subtask termination functions and relax the limitations posed by hand-crafted termination conditions. Experiments in a spatial Search and Rescue domain show that ISEMO can learn the subtask selection policies of various agents grounded in the inter-dependencies among the agents, as well as learn the subtask termination conditions, and perform better than the standard MAHRL technique.

Keywords

Multi-agent Coordination, Reinforcement Learning, search and rescue

Discipline

Artificial Intelligence and Robotics | Databases and Information Systems

Research Areas

Data Science and Engineering

Publication

Proceedings of 2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019, Xiamen, China, December 6-9

First Page

86

Last Page

93

ISBN

9781728124858

Identifier

10.1109/SSCI44817.2019.9002777

Publisher

Institute of Electrical and Electronics Engineers Inc.

City or Country

Xiamen

This document is currently not available here.

Share

COinS