Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

10-2012

Abstract

The basic tenet of a learning process is for an agent to learn for only as much and as long as it is necessary. With reinforcement learning, the learning process is divided between exploration and exploitation. Given the complexity of the problem domain and the randomness of the learning process, the exact duration of the reinforcement learning process can never be known with certainty. Using an inaccurate number of training iterations leads either to the non-convergence or the over-training of the learning agent. This work addresses such issues by proposing a technique to self-regulate the exploration rate and training duration leading to convergence efficiently. The idea originates from an intuitive understanding that exploration is only necessary when the success rate is low. This means the rate of exploration should be conducted in inverse proportion to the rate of success. In addition, the change in exploration-exploitation rates alters the duration of the learning process. Using this approach, the duration of the learning process becomes adaptive to the updated status of the learning process. Experimental results from the K-Armed Bandit and Air Combat Maneuver scenario prove that optimal action policies can be discovered using the right amount of training iterations. In essence, the proposed method eliminates the guesswork on the amount of exploration needed during reinforcement learning.

Keywords

reinforcement learning, exploration-exploitation dilemma, k-armed bandit, pursuit-evasion, self-organizing neuralnetwork

Discipline

Artificial Intelligence and Robotics | Numerical Analysis and Scientific Computing

Research Areas

Data Science and Engineering; Intelligent Systems and Optimization

Publication

Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT 2012)

Volume

13

Identifier

10.1016/j.procs.2012.09.110

City or Country

Bangkok, Thailand

Download

Share

COinS