Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2009
Abstract
We consider the task of developing an adaptive autonomous agent that can interact with non-stationary environments. Traditional learning approaches such as Reinforcement Learning assume stationary characteristics over the course of the problem, and are therefore unable to learn the dynamically changing settings correctly. We introduce a novel adaptive framework that can detect dynamic changes due to non-stationary elements. The Surprise Triggered Adaptive and Reactive (STAR) framework is inspired by human adaptability in dealing with daily life changes. An agent adopting the STAR framework consists primarily of two components, Adapter and Reactor. The Reactor chooses suitable actions based on predictions made by a model of the environment. The Adapter observes the amount of "surprisingness" and triggers the generation of new models accordingly. Preliminary experimental results show that STAR agents are competitive in performance as compared with current approaches, while being much more costeffective by avoiding the negative effects of historical data. Furthermore, since response and adaptability are decoupled in the framework, the adaptive component can benefit other autonomous agents in a variety of domains with nonstationary environments.© 2009, Association for the Advancement of Artificial.
Discipline
Artificial Intelligence and Robotics | Numerical Analysis and Scientific Computing
Publication
Proceedings of the Fifth Artificial Intelligence and Interactive Digital Entertainment Conference, AIIDE 2009, October 14-16, 2009, Stanford
First Page
82
Last Page
87
ISBN
9781577354314
Publisher
AAAI Press
City or Country
Menlo Park, CA
Citation
NGUYEN, Truong-Huy Dinh and Tze-Yun LEONG.
A Surprise Triggered Adaptive and Reactive (STAR) Framework for Online Adaptation in Non-stationary Environments. (2009). Proceedings of the Fifth Artificial Intelligence and Interactive Digital Entertainment Conference, AIIDE 2009, October 14-16, 2009, Stanford. 82-87.
Available at: https://ink.library.smu.edu.sg/sis_research/2993
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
http://www.aaai.org/ocs/index.php/AIIDE/AIIDE09/paper/viewFile/817/1079
Included in
Artificial Intelligence and Robotics Commons, Numerical Analysis and Scientific Computing Commons