MindAgent: Emergent Gaming Interaction
Large Language Models (LLMs) have the capacity of performing complex
scheduling in a multi-agent system and can coordinate these agents into
completing sophisticated tasks that require extensive collaboration. However,
despite the introduction of numerous gaming frameworks, the community has
insufficient benchmarks towards building general multi-agents collaboration
infrastructure that encompass both LLM and human-NPCs collaborations. In this
work, we propose a novel infrastructure - MindAgent - to evaluate planning and
coordination emergent capabilities for gaming interaction. In particular, our
infrastructure leverages existing gaming framework, to i) require understanding
of the coordinator for a multi-agent system, ii) collaborate with human players
via un-finetuned proper instructions, and iii) establish an in-context learning
on few-shot prompt with feedback. Furthermore, we introduce CUISINEWORLD, a new
gaming scenario and related benchmark that dispatch a multi-agent collaboration
efficiency and supervise multiple agents playing the game simultaneously. We
conduct comprehensive evaluations with new auto-metric CoS for calculating the
collaboration efficiency. Finally, our infrastructure can be deployed into
real-world gaming scenarios in a customized VR version of CUISINEWORLD and
adapted in existing broader Minecraft gaming domain. We hope our findings on
LLMs and the new infrastructure for general-purpose scheduling and coordination
can help shed light on how such skills can be obtained by learning from large
language corpora.
650914e6948ce5dce89c8b81
Ran Gong
admin_assigned
2023-09-20T14:24:15.670Z
64acdaf75c6fc404f0d95e5b
/avatars/a4235fb6984e5945eb0e11c3ce581b59.svg
Ran Gong
user
nikepupu
650914e6948ce5dce89c8b82
Qiuyuan Huang
650914e6948ce5dce89c8b83
Xiaojian Ma
admin_assigned
2023-09-20T14:24:38.100Z
60dd0e36a15ddd7d2006d2e9
/avatars/8bd98177a79efbf295be8f6457683297.svg
Xiaojian Ma
user
jeasinema
650914e6948ce5dce89c8b84
Hoi Vo
admin_assigned
2023-09-20T14:24:47.429Z
631aae9bfac58c9c816636e1
/avatars/92d030f9eb552a334aa129e02d3f2ca6.svg
Hoi Vo
user
Hoivo63
650914e6948ce5dce89c8b85
Zane Durante
admin_assigned
2023-09-20T14:24:54.528Z
650259fda33ce3ae28620bf3
/avatars/5fb455b2e3a9523f07230d81d1947e15.svg
Zane Durante
user
ZaneDurante
650914e6948ce5dce89c8b86
Yusuke Noda
admin_assigned
2023-09-20T14:25:02.750Z
6430f78767688e5e12a408e4
/avatars/a7a10d30123495a3eb87d944ae9f2b55.svg
Yusuke Noda
user
yusukenoda
650914e6948ce5dce89c8b87
Zilong Zheng
admin_assigned
2023-09-20T14:25:09.855Z
63a95a6a7930fa8c7dd63d4e
/avatars/d9d0420f7ddfe2f3a7e029fb05f1c89f.svg
Zilong Zheng
user
zlzheng
650914e6948ce5dce89c8b88
Song-Chun Zhu
650914e6948ce5dce89c8b89
Demetri Terzopoulos
650914e6948ce5dce89c8b8a
Li Fei-Fei
650914e6948ce5dce89c8b8b
Jianfeng Gao
admin_assigned
2023-09-20T14:26:08.207Z
641904caf9d6f1d772ec7af7
/avatars/4a63eac71eb30f70b1a0e9d4708f26c1.svg
Jianfeng Gao
user
wyngjf
MindAgent: Emergent Gaming Interaction
Large Language Models (LLMs) have the capacity of performing complex
scheduling in a multi-agent system and can coordinate these agents into
completing sophisticated tasks that require extensive collaboration. However,
despite the introduction of numerous gaming frameworks, the community has
insufficient benchmarks towards building general multi-agents collaboration
infrastructure that encompass both LLM and human-NPCs collaborations. In this
work, we propose a novel infrastructure - MindAgent - to evaluate planning and
coordination emergent capabilities for gaming interaction. In particular, our
infrastructure leverages existing gaming framework, to i) require understanding
of the coordinator for a multi-agent system, ii) collaborate with human players
via un-finetuned proper instructions, and iii) establish an in-context learning
on few-shot prompt with feedback. Furthermore, we introduce CUISINEWORLD, a new
gaming scenario and related benchmark that dispatch a multi-agent collaboration
efficiency and supervise multiple agents playing the game simultaneously. We
conduct comprehensive evaluations with new auto-metric CoS for calculating the
collaboration efficiency. Finally, our infrastructure can be deployed into
real-world gaming scenarios in a customized VR version of CUISINEWORLD and
adapted in existing broader Minecraft gaming domain. We hope our findings on
LLMs and the new infrastructure for general-purpose scheduling and coordination
can help shed light on how such skills can be obtained by learning from large
language corpora.
MindAgent infrastructure evaluates multi-agent coordination and collaboration using LLMs in gaming scenarios, introducing CUISINEWORLD and CoS metric.
Large Language Models LLMs multi-agent system coordination infrastructure MindAgent gaming framework un-finetuned instructions in-context learning few-shot prompt CUISINEWORLD collaboration efficiency benchmark auto-metric CoS VR Minecraft learning from large language corpora