Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

5-2023

Abstract

Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample inefficiency.Generally, episodic control-based approaches are solutions that leveragehighly-rewarded past experiences to improve sample efficiency of DRL algorithms.However, previous episodic control-based approaches fail to utilize the latentinformation from the historical behaviors (e.g., state transitions, topological similarities,etc.) and lack scalability during DRL training. This work introducesNeural Episodic Control with State Abstraction (NECSA), a simple but effectivestate abstraction-based episodic control containing a more comprehensive episodicmemory, a novel state evaluation, and a multi-step state analysis. We evaluate ourapproach to the MuJoCo and Atari tasks in OpenAI gym domains. The experimentalresults indicate that NECSA achieves higher sample efficiency than thestate-of-the-art episodic control-based approaches. Our data and code are availableat the project website.

Discipline

Artificial Intelligence and Robotics

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of the 11th International Conference on Learning Representations, Kigali, Rwanda, 2023 May 1-5

Publisher

The 11th International Conference on Learning Representations, Kigali, Rwanda, 2023 May 1-5

City or Country

ICLR

Share

COinS