Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
Publication Type
Conference Proceeding Article
Publication Date
12-2019
Abstract
In the complex multi-agent tasks, various agents must cooperate to distribute relevant subtasks among each other to achieve joint task objectives. An agent's choice of the relevant subtask changes over time with the changes in the task environment state. Multi-agent Hierarchical Reinforcement Learning (MAHRL) provides an approach for learning to select the subtasks in response to the environment states, by using the joint task rewards to train various agents. When the joint task involves complex inter-agent dependencies, only a subset of agents might be capable of reaching the rewarding task states while other agents take precursory or intermediate roles. The delayed task reward might not be sufficient in such tasks to learn the coordinating policies for various agents. In this paper, we introduce a novel approach of MAHRL called Inter-Subtask Empowerment based Multi-agent Options (ISEMO) in which an Inter-Subtask Empowerment Reward (ISER) is given to an agent which enables the precondition(s) of other agents' subtasks. ISER is given in addition to the domain task reward in order to improve the inter-agent coordination. ISEMO also incorporates options model that can learn parameterized subtask termination functions and relax the limitations posed by hand-crafted termination conditions. Experiments in a spatial Search and Rescue domain show that ISEMO can learn the subtask selection policies of various agents grounded in the inter-dependencies among the agents, as well as learn the subtask termination conditions, and perform better than the standard MAHRL technique.
Keywords
Multi-agent Coordination, Reinforcement Learning, search and rescue
Discipline
Artificial Intelligence and Robotics | Databases and Information Systems
Research Areas
Data Science and Engineering
Publication
Proceedings of 2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019, Xiamen, China, December 6-9
First Page
86
Last Page
93
ISBN
9781728124858
Identifier
10.1109/SSCI44817.2019.9002777
Publisher
Institute of Electrical and Electronics Engineers Inc.
City or Country
Xiamen
Citation
1