Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
5-2023
Abstract
Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample inefficiency.Generally, episodic control-based approaches are solutions that leveragehighly-rewarded past experiences to improve sample efficiency of DRL algorithms.However, previous episodic control-based approaches fail to utilize the latentinformation from the historical behaviors (e.g., state transitions, topological similarities,etc.) and lack scalability during DRL training. This work introducesNeural Episodic Control with State Abstraction (NECSA), a simple but effectivestate abstraction-based episodic control containing a more comprehensive episodicmemory, a novel state evaluation, and a multi-step state analysis. We evaluate ourapproach to the MuJoCo and Atari tasks in OpenAI gym domains. The experimentalresults indicate that NECSA achieves higher sample efficiency than thestate-of-the-art episodic control-based approaches. Our data and code are availableat the project website.
Discipline
Artificial Intelligence and Robotics
Research Areas
Intelligent Systems and Optimization
Publication
Proceedings of the 11th International Conference on Learning Representations, Kigali, Rwanda, 2023 May 1-5
Publisher
The 11th International Conference on Learning Representations, Kigali, Rwanda, 2023 May 1-5
City or Country
ICLR
Citation
LI, Zhuo; ZHU, Derui; HU, Yujing; XIE, Xiaofei; MA, Lei; ZHENG, Yan; SONG, Yan; CHEN, Yingfeng; and ZHAO, Jianjun.
Neural episodic control with state abstraction. (2023). Proceedings of the 11th International Conference on Learning Representations, Kigali, Rwanda, 2023 May 1-5.
Available at: https://ink.library.smu.edu.sg/sis_research/8231
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.