Publication Type

Journal Article

Version

publishedVersion

Publication Date

6-2026

Abstract

Multi-agent reinforcement learning (MARL) empowers multiple autonomous agents to acquire effective policies for collaborative problem-solving. Over the last decade, MARL has seen significant advancements, with numerous algorithms achieving impressive performance across various benchmarks and real-world applications. Nevertheless, the scalability of multi-agent systems, in terms of the number of agents and the length of the task horizon, remains a critical consideration for applying MARL methods to complex problem-solving. Given that a dedicated review of the existing approaches and challenges in scaling up multi-agent systems remains largely absent, this survey aims to bridge this gap by delivering a comprehensive review of MARL methods developed to tackle challenging, scaled-up tasks. To this end, a novel taxonomy of MARL studies is introduced, categorizing them based on the external organizational control structures over all agents and the internal policy structures of individual agents. The survey also discusses the scales of popular MARL environments and tasks, providing a snapshot of the current challenging problems of interest. Furthermore, this survey underscores a set of critical open problems that call for further investigation in the field of scalable MARL.

Keywords

Multi-agent reinforcement learning, Scaling up MARL, Long-horizon

Discipline

Artificial Intelligence and Robotics

Publication

ACM Computing Surveys

Volume

58

Issue

14

First Page

1

Last Page

37

ISSN

0360-0300

Identifier

10.1145/3817113

Publisher

Association for Computing Machinery (ACM)

Additional URL

https://doi-org.libproxy.smu.edu.sg/10.1145/3817113

Download

Share

COinS