Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

12-2024

Abstract

Unsupervised Environment Design (UED) formalizes the problem of autocurricula through interactive training between a teacher agent and a student agent. The teacher generates new training environments with high learning potential, curating an adaptive curriculum that strengthens the student’s ability to handle unseen scenarios. Existing UED methods mainly rely on regret, a metric that measures the difference between the agent’s optimal and actual performance, to guide curriculum design. Regret-driven methods generate curricula that progressively increase environment complexity for the student but overlook environment novelty–a critical element for enhancing an agent’s generalizability. Measuring environment novelty is especially challenging due to the underspecified nature of environment parameters in UED, and existing approaches face significant limitations. To address this, this paper introduces the Coverage-based Evaluation of Novelty In Environment (CENIE) framework. CENIE proposes a scalable, domainagnostic, and curriculum-aware approach to quantifying environment novelty by leveraging the student’s state-action space coverage from previous curriculum experiences. We then propose an implementation of CENIE that models this coverage and measures environment novelty using Gaussian Mixture Models. By integrating both regret and novelty as complementary objectives for curriculum design, CENIE facilitates effective exploration across the state-action space while progressively increasing curriculum complexity. Empirical evaluations demonstrate that augmenting existing regret-based UED algorithms with CENIE achieves stateof-the-art performance across multiple benchmarks, underscoring the effectiveness of novelty-driven autocurricula for robust generalization.

Keywords

Unsupervised environment design, Regret-driven metric, Environment novelty

Discipline

Artificial Intelligence and Robotics | Databases and Information Systems | Educational Assessment, Evaluation, and Research

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024) : Vancouver, Canada, December 10-15

First Page

1

Last Page

28

Publisher

NeurIPS

City or Country

Vancouver, Canada

Copyright Owner and License

Authors

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Additional URL

https://neurips.cc/virtual/2024/poster/94954

Download

Share

COinS