Delayed Observation Planning in Partially Observable Domains
Conference Proceeding Article
Traditional models for planning under uncertainty such as Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) assume that the observations about the results of agent actions are instantly available to the agent. In so doing, they are no longer applicable to domains where observations are received with delays caused by temporary unavailability of information (e.g. delayed response of the market to a new product). To that end, we make the following key contributions towards solving Delayed observation POMDPs (D-POMDPs): (i) We first provide an parameterized approximate algorithm for solving D-POMDPs efficiently, with desired accuracy; and (ii) We then propose a policy execution technique that adjusts the policy at run-time to account for the actual realization of observations. We then show the performance of our techniques on POMDP benchmark problems with delayed observations where explicit modeling of delayed observations leads to solutions of superior quality.
Partially Observable Markov Decision Process, Delayed Observations
Artificial Intelligence and Robotics | Business | Operations Research, Systems Engineering and Industrial Engineering
Intelligent Systems and Decision Analytics
Proceedings of the Eleventh International Conference on Autonomous Agents and Multi-Agent Systems, published by ACM
City or Country
VARAKANTHAM, Pradeep Reddy and Marecki, Janusz.
Delayed Observation Planning in Partially Observable Domains. (2012). Proceedings of the Eleventh International Conference on Autonomous Agents and Multi-Agent Systems, published by ACM. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/1606