title: ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and
    Vision-Language Models
summary: Integrating Large Language Models with symbolic planners is a promising
  direction for obtaining verifiable and grounded plans compared to planning in
  natural language, with recent works extending this idea to visual domains using
  Vision-Language Models (VLMs). However, rigorous comparison between
  VLM-grounded symbolic approaches and methods that plan directly with a VLM has
  been hindered by a lack of common environments, evaluation protocols and model
  coverage. We introduce ViPlan, the first open-source benchmark for Visual
  Planning with symbolic predicates and VLMs. ViPlan features a series of
  increasingly challenging tasks in two domains: a visual variant of the classic
  Blocksworld planning problem and a simulated household robotics environment. We
  benchmark nine open-source VLM families across multiple sizes, along with
  selected closed models, evaluating both VLM-grounded symbolic planning and
  using the models directly to propose actions. We find symbolic planning to
  outperform direct VLM planning in Blocksworld, where accurate image grounding
  is crucial, whereas the opposite is true in the household robotics tasks, where
  commonsense knowledge and the ability to recover from errors are beneficial.
  Finally, we show that across most models and methods, there is no significant
  benefit to using Chain-of-Thought prompting, suggesting that current VLMs still
  struggle with visual reasoning.
paper_authors._id: 682c389bc19ea9cd7d822b5c
paper_authors.name: Matteo Merler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:39:09.385Z
paper_authors._id: 644555c72d91b15b4c7ebd1c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg
paper_authors.fullname: Matteo Merler
paper_authors.type: user
paper_authors.user: merlerm
paper_authors._id: 682c389bc19ea9cd7d822b5d
paper_authors.name: Nicola Dainese
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:39:30.830Z
paper_authors._id: 6382346663e3fab40c8c66f9
paper_authors.avatarUrl: /avatars/bcdba23952ff465b8488bd68a61005e5.svg
paper_authors.fullname: Nicola Dainese
paper_authors.type: user
paper_authors.user: dainesn1
paper_authors._id: 682c389bc19ea9cd7d822b5e
paper_authors.name: Minttu Alakuijala
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T08:23:26.845Z
paper_authors._id: 64c268c4b57937d56d65e163
paper_authors.avatarUrl: /avatars/bf290d81983703e457e709fec1a2300e.svg
paper_authors.fullname: Minttu Alakuijala
paper_authors.type: user
paper_authors.user: minttusofia
paper_authors._id: 682c389bc19ea9cd7d822b5f
paper_authors.name: Giovanni Bonetta
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:39:37.830Z
paper_authors._id: 60d9e5b71fa5d458da777550
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676481484908-60d9e5b71fa5d458da777550.png
paper_authors.fullname: Giovanni Bonetta
paper_authors.type: user
paper_authors.user: giobin
paper_authors._id: 682c389bc19ea9cd7d822b60
paper_authors.name: Pietro Ferrazzi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:40:03.086Z
paper_authors._id: 6512b0e48c0f10eedb296c65
paper_authors.avatarUrl: /avatars/46cf7ddf5f94468b7cf39a787741ca2d.svg
paper_authors.fullname: Pietro Ferrazzi
paper_authors.type: user
paper_authors.user: Pietroferr
paper_authors._id: 682c389bc19ea9cd7d822b61
paper_authors.name: Yu Tian
paper_authors._id: 682c389bc19ea9cd7d822b62
paper_authors.name: Bernardo Magnini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:40:11.231Z
paper_authors._id: 666d3b2bb955b0e655473ffe
paper_authors.avatarUrl: /avatars/de83261afe1655b857a34f3c9f1d0bcc.svg
paper_authors.fullname: Bernardo Magnini
paper_authors.type: user
paper_authors.user: magnini
paper_authors._id: 682c389bc19ea9cd7d822b63
paper_authors.name: Pekka Marttinen
paper_title: ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and
    Vision-Language Models
paper_summary: Integrating Large Language Models with symbolic planners is a promising
  direction for obtaining verifiable and grounded plans compared to planning in
  natural language, with recent works extending this idea to visual domains using
  Vision-Language Models (VLMs). However, rigorous comparison between
  VLM-grounded symbolic approaches and methods that plan directly with a VLM has
  been hindered by a lack of common environments, evaluation protocols and model
  coverage. We introduce ViPlan, the first open-source benchmark for Visual
  Planning with symbolic predicates and VLMs. ViPlan features a series of
  increasingly challenging tasks in two domains: a visual variant of the classic
  Blocksworld planning problem and a simulated household robotics environment. We
  benchmark nine open-source VLM families across multiple sizes, along with
  selected closed models, evaluating both VLM-grounded symbolic planning and
  using the models directly to propose actions. We find symbolic planning to
  outperform direct VLM planning in Blocksworld, where accurate image grounding
  is crucial, whereas the opposite is true in the household robotics tasks, where
  commonsense knowledge and the ability to recover from errors are beneficial.
  Finally, we show that across most models and methods, there is no significant
  benefit to using Chain-of-Thought prompting, suggesting that current VLMs still
  struggle with visual reasoning.
paper_ai_summary: ViPlan is an open-source benchmark comparing symbolic planning with Vision-Language Models and direct action proposals in visual domains, finding that symbolic planning excels in tasks requiring accurate image grounding.
paper_ai_keywords: Vision-Language Models VLMs symbolic planning Chain-of-Thought prompting Blocksworld visual planning household robotics image grounding commonsense knowledge error recovery visual reasoning