title: VS-Bench: Evaluating VLMs for Strategic Reasoning and Decision-Making in
    Multi-Agent Environments
summary: Recent advancements in Vision Language Models (VLMs) have expanded their
  capabilities to interactive agent tasks, yet existing benchmarks remain limited
  to single-agent or text-only environments. In contrast, real-world scenarios
  often involve multiple agents interacting within rich visual and linguistic
  contexts, posing challenges with both multimodal observations and strategic
  interactions. To bridge this gap, we introduce Visual Strategic Bench
  (VS-Bench), a multimodal benchmark that evaluates VLMs for strategic reasoning
  and decision-making in multi-agent environments. VS-Bench comprises eight
  vision-grounded environments spanning cooperative, competitive, and
  mixed-motive interactions, designed to assess agents' ability to predict
  others' future moves and optimize for long-term objectives. We consider two
  complementary evaluation dimensions, including offline evaluation of strategic
  reasoning by next-action prediction accuracy and online evaluation of
  decision-making by normalized episode return. Extensive experiments of fourteen
  leading VLMs reveal a significant gap between current models and optimal
  performance, with the best models attaining 47.8% prediction accuracy and 24.3%
  normalized return. We further conduct in-depth analyses on multimodal
  observations, test-time scaling, social behaviors, and failure cases of VLM
  agents. By standardizing the evaluation and highlighting the limitations of
  existing models, we envision VS-Bench as a foundation for future research on
  strategic multimodal agents. Code and data are available at
  https://vs-bench.github.io.
paper_authors._id: 683fa95ea0770843560c7ae3
paper_authors.name: Zelai Xu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-05T12:16:05.641Z
paper_authors._id: 653a5b0f7c01c693a16dd184
paper_authors.avatarUrl: /avatars/4b43d88709dc8037250404452e81adcf.svg
paper_authors.fullname: Zelai Xu
paper_authors.type: user
paper_authors.user: zelaix
paper_authors._id: 683fa95ea0770843560c7ae4
paper_authors.name: Zhexuan Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T12:42:09.925Z
paper_authors._id: 683984ae71d8234f74f2166d
paper_authors.avatarUrl: /avatars/fd8b37f6a12ef24ae37c0d127fc3f695.svg
paper_authors.fullname: xzxuan
paper_authors.type: user
paper_authors.user: xzxuan
paper_authors._id: 683fa95ea0770843560c7ae5
paper_authors.name: Xiangmin Yi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T17:08:16.562Z
paper_authors._id: 67c6fd7dc46aff331d05e49a
paper_authors.avatarUrl: /avatars/a1876ec436f9d15a51c9f5b5ca333f25.svg
paper_authors.fullname: Xiangmin Yi
paper_authors.type: user
paper_authors.user: lazyyxm
paper_authors._id: 683fa95ea0770843560c7ae6
paper_authors.name: Huining Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-04T10:26:15.430Z
paper_authors._id: 683fb7bb7c8d720be437948d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/pWekQrjYM6lw9bDjvLHg7.png
paper_authors.fullname: Huining Yuan
paper_authors.type: user
paper_authors.user: HuiningYuan
paper_authors._id: 683fa95ea0770843560c7ae7
paper_authors.name: Xinlei Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:41:40.125Z
paper_authors._id: 63e58e3a006a775275e59e41
paper_authors.avatarUrl: /avatars/75262a35b27a2ae1939df9118120d99e.svg
paper_authors.fullname: Xinlei Chen
paper_authors.type: user
paper_authors.user: endernewton
paper_authors._id: 683fa95ea0770843560c7ae8
paper_authors.name: Yi Wu
paper_authors._id: 683fa95ea0770843560c7ae9
paper_authors.name: Chao Yu
paper_authors._id: 683fa95ea0770843560c7aea
paper_authors.name: Yu Wang
paper_title: VS-Bench: Evaluating VLMs for Strategic Reasoning and Decision-Making in
    Multi-Agent Environments
paper_summary: Recent advancements in Vision Language Models (VLMs) have expanded their
  capabilities to interactive agent tasks, yet existing benchmarks remain limited
  to single-agent or text-only environments. In contrast, real-world scenarios
  often involve multiple agents interacting within rich visual and linguistic
  contexts, posing challenges with both multimodal observations and strategic
  interactions. To bridge this gap, we introduce Visual Strategic Bench
  (VS-Bench), a multimodal benchmark that evaluates VLMs for strategic reasoning
  and decision-making in multi-agent environments. VS-Bench comprises eight
  vision-grounded environments spanning cooperative, competitive, and
  mixed-motive interactions, designed to assess agents' ability to predict
  others' future moves and optimize for long-term objectives. We consider two
  complementary evaluation dimensions, including offline evaluation of strategic
  reasoning by next-action prediction accuracy and online evaluation of
  decision-making by normalized episode return. Extensive experiments of fourteen
  leading VLMs reveal a significant gap between current models and optimal
  performance, with the best models attaining 47.8% prediction accuracy and 24.3%
  normalized return. We further conduct in-depth analyses on multimodal
  observations, test-time scaling, social behaviors, and failure cases of VLM
  agents. By standardizing the evaluation and highlighting the limitations of
  existing models, we envision VS-Bench as a foundation for future research on
  strategic multimodal agents. Code and data are available at
  https://vs-bench.github.io.
paper_ai_summary: VS-Bench is a multimodal benchmark designed to evaluate Vision Language Models' strategic reasoning and decision-making in complex multi-agent environments.
paper_ai_keywords: Vision Language Models VS-Bench multimodal benchmark strategic reasoning decision-making multi-agent environments vision-grounded environments cooperative competitive mixed-motive interactions next-action prediction normalized episode return multimodal observations test-time scaling social behaviors failure cases