Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

8-2023

Abstract

Agent decision making using Reinforcement Learning (RL) heavily relies on either a model or simulator of the environment (e.g., moving in an 8x8 maze with three rooms, playing Chess on an 8x8 board). Due to this dependence, small changes in the environment (e.g., positions of obstacles in the maze, size of the board) can severely affect the effectiveness of the policy learned by the agent. To that end, existing work has proposed training RL agents on an adaptive curriculum of environments (generated automatically) to improve performance on out-of-distribution (OOD) test scenarios. Specifically, existing research has employed the potential for the agent to learn in an environment (captured using Generalized Advantage Estimation, GAE) as the key factor to select the next environment(s) to train the agent. However, such a mechanism can select similar environments (with a high potential to learn) thereby making agent training redundant on all but one of those environments. To that end, we provide a principled approach to adaptively identify diverse environments based on a novel distance measure relevant to environment design. We empirically demonstrate the versatility and effectiveness of our method in comparison to multiple leading approaches for unsupervised environment design on three distinct benchmark problems used in literature.

Keywords

Planning and Scheduling, Search in planning and scheduling, Machine Learning, Deep reinforcement learning

Discipline

Artificial Intelligence and Robotics | Operations Research, Systems Engineering and Industrial Engineering

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of the 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023: Macao, August 19-25

First Page

5411

Last Page

5419

ISBN

9781956792034

Identifier

10.24963/ijcai.2023/601

Publisher

AAAI Press

City or Country

Washington, DC

Additional URL

https://doi.org/10.24963/ijcai.2023/601

Share

COinS