Publication Type
Journal Article
Version
acceptedVersion
Publication Date
6-2021
Abstract
Hierarchical Reinforcement Learning (HRL) enables autonomous decomposition of challenging long-horizon decision-making tasks into simpler subtasks. During the past years, the landscape of HRL research has grown profoundly, resulting in copious approaches. A comprehensive overview of this vast landscape is necessary to study HRL in an organized manner. We provide a survey of the diverse HRL approaches concerning the challenges of learning hierarchical policies, subtask discovery, transfer learning, and multi-agent learning using HRL. The survey is presented according to a novel taxonomy of the approaches. Based on the survey, a set of important open problems is proposed to motivate the future research in HRL. Furthermore, we outline a few suitable task domains for evaluating the HRL approaches and a few interesting examples of the practical applications of HRL in the Supplementary Material.
Keywords
Hierarchical reinforcement learning, subtask discovery, skill discovery, hierarchical reinforcement learning survey, hierarchical reinforcement learning taxonomy
Discipline
Databases and Information Systems | Data Storage Systems
Research Areas
Data Science and Engineering
Publication
ACM Computing Surveys
Volume
54
Issue
5
First Page
1
Last Page
35
ISSN
0360-0300
Identifier
10.1145/3453160
Publisher
Association for Computing Machinery (ACM)
Embargo Period
8-3-2021
Citation
PATERIA, Shubham; SUBAGDJA, Budhitama; TAN, Ah-hwee; and QUEK, Chai.
Hierarchical reinforcement learning: A comprehensive survey. (2021). ACM Computing Surveys. 54, (5), 1-35.
Available at: https://ink.library.smu.edu.sg/sis_research/6047
Copyright Owner and License
Author
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3453160