Publication Type

Journal Article

Version

publishedVersion

Publication Date

2-2008

Abstract

This paper presents a neural architecture for learning category nodes encoding mappings across multimodal patterns involving sensory inputs, actions, and rewards. By integrating adaptive resonance theory (ART) and temporal difference (TD) methods, the proposed neural model, called TD fusion architecture for learning, cognition, and navigation (TD-FALCON), enables an autonomous agent to adapt and function in a dynamic environment with immediate as well as delayed evaluative feedback (reinforcement) signals. TD-FALCON learns the value functions of the state-action space estimated through on-policy and off-policy TD learning methods, specifically state-action-reward-state-action (SARSA) and Q-learning. The learned value functions are then used to determine the optimal actions based on an action selection policy. We have developed TD-FALCON systems using various TD learning strategies and compared their performance in terms of task completion, learning speed, as well as time and space efficiency. Experiments based on a minefield navigation task have shown that TD-FALCON systems are able to learn effectively with both immediate and delayed reinforcement and achieve a stable performance in a pace much faster than those of standard gradient-descent-based reinforcement learning systems.

Keywords

Reinforcement learning, self-organizing neural networks (NNs), temporal difference (TD) methods

Discipline

Computer Engineering | Databases and Information Systems | OS and Networks

Research Areas

Data Science and Engineering

Publication

IEEE Transactions on Neural Networks

Volume

9

Issue

2

First Page

230

Last Page

244

ISSN

1045-9227

Identifier

10.1109/TNN.2007.905839

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Additional URL

https://doi.org/10.1109/TNN.2007.905839

Download

Share

COinS