Conference Proceeding Article
Researchers have introduced the Dynamic Distributed Constraint Optimization Problem (Dynamic DCOP) formulation to model dynamically changing multi-agent coordination problems, where a dynamic DCOP is a sequence of (static canonical) DCOPs, each partially different from the DCOP preceding it. Existing work typically assumes that the problem in each time step is decoupled from the problems in other time steps, which might not hold in some applications. Therefore, in this paper, we make the following contributions: (i) We introduce a new model, called Markovian Dynamic DCOPs (MD-DCOPs), where the DCOP in the next time step is a function of the value assignments in the current time step; (ii) We introduce two distributed reinforcement learning algorithms, the Distributed RVI Q-learning algorithm and the Distributed R-learning algorithm, that balance exploration and exploitation to solve MD-DCOPs in an online manner; and (iii) We empirically evaluate them against an existing multiarm bandit DCOP algorithm on dynamic DCOPs.
Artificial Intelligence and Robotics | Operations Research, Systems Engineering and Industrial Engineering
Intelligent Systems and Decision Analytics
AAMAS '14: Proceedings of the 2014 International Conference on Autonomous Agents and Multiagent Systems: May 5-9, 2014, Paris, France
City or Country
NGUYEN, Duc Thien; YEOH, William; LAU, Hoong Chuin; and Zilberstein, Shlomo.
Decentralized multi-agent reinforcement learning in average-reward dynamic DCOPs. (2014). AAMAS '14: Proceedings of the 2014 International Conference on Autonomous Agents and Multiagent Systems: May 5-9, 2014, Paris, France. 1341-1342. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/2009
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.