title: Visual Planning: Let's Think Only with Images
summary: Recent advancements in Large Language Models (LLMs) and their multimodal
  extensions (MLLMs) have substantially enhanced machine reasoning across diverse
  tasks. However, these models predominantly rely on pure text as the medium for
  both expressing and structuring reasoning, even when visual information is
  present. In this work, we argue that language may not always be the most
  natural or effective modality for reasoning, particularly in tasks involving
  spatial and geometrical information. Motivated by this, we propose a new
  paradigm, Visual Planning, which enables planning through purely visual
  representations, independent of text. In this paradigm, planning is executed
  via sequences of images that encode step-by-step inference in the visual
  domain, akin to how humans sketch or visualize future actions. We introduce a
  novel reinforcement learning framework, Visual Planning via Reinforcement
  Learning (VPRL), empowered by GRPO for post-training large vision models,
  leading to substantial improvements in planning in a selection of
  representative visual navigation tasks, FrozenLake, Maze, and MiniBehavior. Our
  visual planning paradigm outperforms all other planning variants that conduct
  reasoning in the text-only space. Our results establish Visual Planning as a
  viable and promising alternative to language-based reasoning, opening new
  avenues for tasks that benefit from intuitive, image-based inference.
paper_authors._id: 682abb7984695084c1a48eab
paper_authors.name: Yi Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:23:00.135Z
paper_authors._id: 644d55bd53ad80c659395473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644d55bd53ad80c659395473/Jdff2RyL6ZGhFSY8kab1L.jpeg
paper_authors.fullname: Yi Xu
paper_authors.type: user
paper_authors.user: yixu1
paper_authors._id: 682abb7984695084c1a48eac
paper_authors.name: Chengzu Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-19T13:09:14.188Z
paper_authors._id: 650e9b0288cdfe73a8575923
paper_authors.avatarUrl: /avatars/0fc7fcd0776f63ea5f50a310e7def2f5.svg
paper_authors.fullname: Chengzu Li
paper_authors.type: user
paper_authors.user: chengzu
paper_authors._id: 682abb7984695084c1a48ead
paper_authors.name: Han Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-19T06:42:16.276Z
paper_authors._id: 62b279e92375526ae51a537b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b279e92375526ae51a537b/U2DxDscDjQ6kWh-jMn0IG.jpeg
paper_authors.fullname: Han Zhou
paper_authors.type: user
paper_authors.user: hzhouml
paper_authors._id: 682abb7984695084c1a48eae
paper_authors.name: Xingchen Wan
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-19T05:02:52.536Z
paper_authors._id: 65bf213f8467e2a3d6374d4b
paper_authors.avatarUrl: /avatars/0194cdba95d7a4c01fbbdd505e384a3d.svg
paper_authors.fullname: X Wan
paper_authors.type: user
paper_authors.user: masonxw
paper_authors._id: 682abb7984695084c1a48eaf
paper_authors.name: Caiqi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-19T07:23:48.005Z
paper_authors._id: 63920dfac47e36ddeb8f1864
paper_authors.avatarUrl: /avatars/c36cbf7b084d62368312e5c9292e4260.svg
paper_authors.fullname: Caiqi Zhang
paper_authors.type: user
paper_authors.user: caiqizh
paper_authors._id: 682abb7984695084c1a48eb0
paper_authors.name: Anna Korhonen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-19T07:23:42.059Z
paper_authors._id: 617a6284941993035fbaf299
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1635410461794-noauth.jpeg
paper_authors.fullname: Anna Korhonen
paper_authors.type: user
paper_authors.user: akorhonen
paper_authors._id: 682abb7984695084c1a48eb1
paper_authors.name: Ivan Vulić
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-19T07:23:36.111Z
paper_authors._id: 6273e70dc8d55dd434bd8e52
paper_authors.avatarUrl: /avatars/3483eeda218e95b1eb00c3dc63c7d000.svg
paper_authors.fullname: Ivan Vulić
paper_authors.type: user
paper_authors.user: ivulic
paper_title: Visual Planning: Let's Think Only with Images
paper_summary: Recent advancements in Large Language Models (LLMs) and their multimodal
  extensions (MLLMs) have substantially enhanced machine reasoning across diverse
  tasks. However, these models predominantly rely on pure text as the medium for
  both expressing and structuring reasoning, even when visual information is
  present. In this work, we argue that language may not always be the most
  natural or effective modality for reasoning, particularly in tasks involving
  spatial and geometrical information. Motivated by this, we propose a new
  paradigm, Visual Planning, which enables planning through purely visual
  representations, independent of text. In this paradigm, planning is executed
  via sequences of images that encode step-by-step inference in the visual
  domain, akin to how humans sketch or visualize future actions. We introduce a
  novel reinforcement learning framework, Visual Planning via Reinforcement
  Learning (VPRL), empowered by GRPO for post-training large vision models,
  leading to substantial improvements in planning in a selection of
  representative visual navigation tasks, FrozenLake, Maze, and MiniBehavior. Our
  visual planning paradigm outperforms all other planning variants that conduct
  reasoning in the text-only space. Our results establish Visual Planning as a
  viable and promising alternative to language-based reasoning, opening new
  avenues for tasks that benefit from intuitive, image-based inference.
paper_ai_summary: Visual Planning, using sequences of images for planning, outperforms text-only reasoning methods in visual navigation tasks.
paper_ai_keywords: Large Language Models multimodal extensions Visual Planning reinforcement learning VPRL GRPO visual navigation tasks FrozenLake Maze MiniBehavior