Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
8-2023
Abstract
Agent decision making using Reinforcement Learning (RL) heavily relies on either a model or simulator of the environment (e.g., moving in an 8x8 maze with three rooms, playing Chess on an 8x8 board). Due to this dependence, small changes in the environment (e.g., positions of obstacles in the maze, size of the board) can severely affect the effectiveness of the policy learned by the agent. To that end, existing work has proposed training RL agents on an adaptive curriculum of environments (generated automatically) to improve performance on out-of-distribution (OOD) test scenarios. Specifically, existing research has employed the potential for the agent to learn in an environment (captured using Generalized Advantage Estimation, GAE) as the key factor to select the next environment(s) to train the agent. However, such a mechanism can select similar environments (with a high potential to learn) thereby making agent training redundant on all but one of those environments. To that end, we provide a principled approach to adaptively identify diverse environments based on a novel distance measure relevant to environment design. We empirically demonstrate the versatility and effectiveness of our method in comparison to multiple leading approaches for unsupervised environment design on three distinct benchmark problems used in literature.
Keywords
Planning and Scheduling, Search in planning and scheduling, Machine Learning, Deep reinforcement learning
Discipline
Artificial Intelligence and Robotics | Operations Research, Systems Engineering and Industrial Engineering
Research Areas
Intelligent Systems and Optimization
Publication
Proceedings of the 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023: Macao, August 19-25
First Page
5411
Last Page
5419
ISBN
9781956792034
Identifier
10.24963/ijcai.2023/601
Publisher
AAAI Press
City or Country
Washington, DC
Citation
LI, Wenjun; VARAKANTHAM, Pradeep; and LI, Dexun.
Generalization through diversity: Improving unsupervised environment design. (2023). Proceedings of the 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023: Macao, August 19-25. 5411-5419.
Available at: https://ink.library.smu.edu.sg/sis_research/8099
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.24963/ijcai.2023/601
Included in
Artificial Intelligence and Robotics Commons, Operations Research, Systems Engineering and Industrial Engineering Commons