MultiAgentBench: Evaluating the Collaboration and Competition of LLM
  agents
Large Language Models (LLMs) have shown remarkable capabilities as autonomous
agents, yet existing benchmarks either focus on single-agent tasks or are
confined to narrow domains, failing to capture the dynamics of multi-agent
coordination and competition. In this paper, we introduce MultiAgentBench, a
comprehensive benchmark designed to evaluate LLM-based multi-agent systems
across diverse, interactive scenarios. Our framework measures not only task
completion but also the quality of collaboration and competition using novel,
milestone-based key performance indicators. Moreover, we evaluate various
coordination protocols (including star, chain, tree, and graph topologies) and
innovative strategies such as group discussion and cognitive planning. Notably,
gpt-4o-mini reaches the average highest task score, graph structure performs
the best among coordination protocols in the research scenario, and cognitive
planning improves milestone achievement rates by 3%. Code and datasets are
public available at https://github.com/MultiagentBench/MARBLE.
67c7ba7f19b236e0564d1172
Kunlun Zhu
extracted_pending
2025-03-05T02:44:18.739Z
66554507e6ea63012f35824c
/avatars/b82de75bd60890e7bb524fc3754b131c.svg
Kunlun_Zhu
user
Leozkl
67c7ba7f19b236e0564d1173
Hongyi Du
67c7ba7f19b236e0564d1174
Zhaochen Hong
admin_assigned
2025-03-05T10:38:26.340Z
672516b24c0a2bf36e52ec9d
/avatars/ce20ec30586dd40d263fd0b88eaf002d.svg
Zhaochen Hong
user
zhaochenhong
67c7ba7f19b236e0564d1175
Xiaocheng Yang
admin_assigned
2025-03-05T10:38:17.378Z
64f98e9c31b3625c3844b73d
https://cdn-avatars.huggingface.co/v1/production/uploads/64f98e9c31b3625c3844b73d/if9kvZUkXTuo9UuzJY236.jpeg
Xiaocheng Yang
user
Thomas-X-Yang
67c7ba7f19b236e0564d1176
Shuyi Guo
admin_assigned
2025-03-05T10:38:47.235Z
65ca8d92d5822659ecc6c0d4
/avatars/6e9ab0470d68248a267eec892fce89c3.svg
Shuyi
user
ShuyiGuo
67c7ba7f19b236e0564d1177
Zhe Wang
67c7ba7f19b236e0564d1178
Zhenhailong Wang
admin_assigned
2025-03-05T10:34:51.267Z
628d7265db4cd1d1717c884f
/avatars/dff2a3dd10d84b4a73fa486402de7219.svg
Zhenhailong Wang
user
mikewang
67c7ba7f19b236e0564d1179
Cheng Qian
claimed_verified
2025-04-23T08:38:48.207Z
665e121c6007027038fd4005
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/sIVBJAGM-Kneq9KMf8aXb.png
Cheng Qian
user
chengq9
67c7ba7f19b236e0564d117a
Xiangru Tang
admin_assigned
2025-03-05T10:35:40.308Z
63357c608adfa81faf2ac180
/avatars/ae0314c644f882251baf59b9134fd36f.svg
Xiangru Tang
user
RTT1
67c7ba7f19b236e0564d117b
Heng Ji
67c7ba7f19b236e0564d117c
Jiaxuan You
MultiAgentBench: Evaluating the Collaboration and Competition of LLM
  agents
Large Language Models (LLMs) have shown remarkable capabilities as autonomous
agents, yet existing benchmarks either focus on single-agent tasks or are
confined to narrow domains, failing to capture the dynamics of multi-agent
coordination and competition. In this paper, we introduce MultiAgentBench, a
comprehensive benchmark designed to evaluate LLM-based multi-agent systems
across diverse, interactive scenarios. Our framework measures not only task
completion but also the quality of collaboration and competition using novel,
milestone-based key performance indicators. Moreover, we evaluate various
coordination protocols (including star, chain, tree, and graph topologies) and
innovative strategies such as group discussion and cognitive planning. Notably,
gpt-4o-mini reaches the average highest task score, graph structure performs
the best among coordination protocols in the research scenario, and cognitive
planning improves milestone achievement rates by 3%. Code and datasets are
public available at https://github.com/MultiagentBench/MARBLE.
A new benchmark evaluates LLM-based multi-agent systems in diverse scenarios, measuring task completion and quality of collaboration using coordination protocols and innovative strategies.
LLMs autonomous agents MultiAgentBench multi-agent systems milestone-based key performance indicators star chain tree graph topologies group discussion cognitive planning