VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement
  Learning
Coordinating multiple embodied agents in dynamic environments remains a core
challenge in artificial intelligence, requiring both perception-driven
reasoning and scalable cooperation strategies. While recent works have
leveraged large language models (LLMs) for multi-agent planning, a few have
begun to explore vision-language models (VLMs) for visual reasoning. However,
these VLM-based approaches remain limited in their support for diverse
embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical
benchmark tailored for embodied multi-agent cooperation, featuring three
structured levels: agent activation, task planning, and trajectory perception.
VIKI-Bench includes diverse robot embodiments, multi-view visual observations,
and structured supervision signals to evaluate reasoning grounded in visual
inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a
two-stage framework that fine-tunes a pretrained vision-language model (VLM)
using Chain-of-Thought annotated demonstrations, followed by reinforcement
learning under multi-level reward signals. Our extensive experiments show that
VIKI-R significantly outperforms baselines method across all task levels.
Furthermore, we show that reinforcement learning enables the emergence of
compositional cooperation patterns among heterogeneous agents. Together,
VIKI-Bench and VIKI-R offer a unified testbed and method for advancing
multi-agent, visual-driven cooperation in embodied AI systems.
6858c341c0c8e29df8ea3c7f
Li Kang
claimed_verified
2025-06-23T08:14:50.165Z
64eadcb03d76028d805a7818
/avatars/528e4fded4419caf08589b2ed40437bc.svg
Li Kang
user
FACEONG
6858c341c0c8e29df8ea3c80
Xiufeng Song
6858c341c0c8e29df8ea3c81
Heng Zhou
6858c341c0c8e29df8ea3c82
Yiran Qin
6858c341c0c8e29df8ea3c83
Jie Yang
6858c341c0c8e29df8ea3c84
Xiaohong Liu
6858c341c0c8e29df8ea3c85
Philip Torr
6858c341c0c8e29df8ea3c86
Lei Bai
6858c341c0c8e29df8ea3c87
Zhenfei Yin
VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement
  Learning
Coordinating multiple embodied agents in dynamic environments remains a core
challenge in artificial intelligence, requiring both perception-driven
reasoning and scalable cooperation strategies. While recent works have
leveraged large language models (LLMs) for multi-agent planning, a few have
begun to explore vision-language models (VLMs) for visual reasoning. However,
these VLM-based approaches remain limited in their support for diverse
embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical
benchmark tailored for embodied multi-agent cooperation, featuring three
structured levels: agent activation, task planning, and trajectory perception.
VIKI-Bench includes diverse robot embodiments, multi-view visual observations,
and structured supervision signals to evaluate reasoning grounded in visual
inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a
two-stage framework that fine-tunes a pretrained vision-language model (VLM)
using Chain-of-Thought annotated demonstrations, followed by reinforcement
learning under multi-level reward signals. Our extensive experiments show that
VIKI-R significantly outperforms baselines method across all task levels.
Furthermore, we show that reinforcement learning enables the emergence of
compositional cooperation patterns among heterogeneous agents. Together,
VIKI-Bench and VIKI-R offer a unified testbed and method for advancing
multi-agent, visual-driven cooperation in embodied AI systems.
VIKI-Bench and VIKI-R provide a benchmark and framework for evaluating and improving visual-driven cooperation among diverse embodied agents using vision-language models and reinforcement learning.
embodied agents VIKI-Bench multi-agent cooperation vision-language models VIKI-R Chain-of-Thought reinforcement learning compositional cooperation multi-level reward signals robot embodiments multi-view visual observations structured supervision signals