title: World Modeling Makes a Better Planner: Dual Preference Optimization for
    Embodied Task Planning
summary: Recent advances in large vision-language models (LVLMs) have shown promise
  for embodied task planning, yet they struggle with fundamental challenges like
  dependency constraints and efficiency. Existing approaches either solely
  optimize action selection or leverage world models during inference,
  overlooking the benefits of learning to model the world as a way to enhance
  planning capabilities. We propose Dual Preference Optimization (D^2PO), a new
  learning framework that jointly optimizes state prediction and action selection
  through preference learning, enabling LVLMs to understand environment dynamics
  for better planning. To automatically collect trajectories and stepwise
  preference data without human annotation, we introduce a tree search mechanism
  for extensive exploration via trial-and-error. Extensive experiments on
  VoTa-Bench demonstrate that our D^2PO-based method significantly outperforms
  existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and
  LLaMA-3.2 (11B), achieving superior task success rates with more efficient
  execution paths.
paper_authors._id: 67d38a42d3d16e1166d81bed
paper_authors.name: Siyin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:44.686Z
paper_authors._id: 64c3c631e77ea9f28111172a
paper_authors.avatarUrl: /avatars/495dbb73b69c399bae780da3118e332f.svg
paper_authors.fullname: Siyin Wang
paper_authors.type: user
paper_authors.user: sinwang
paper_authors._id: 67d38a42d3d16e1166d81bee
paper_authors.name: Zhaoye Fei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:05:04.607Z
paper_authors._id: 629ef8544313a7c1dd671130
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/629ef8544313a7c1dd671130/i5xfHIgELcuO1Ew19ebTw.png
paper_authors.fullname: Zhaoye Fei
paper_authors.type: user
paper_authors.user: ngc7293
paper_authors._id: 67d38a42d3d16e1166d81bef
paper_authors.name: Qinyuan Cheng
paper_authors._id: 67d38a42d3d16e1166d81bf0
paper_authors.name: Shiduo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:05:20.440Z
paper_authors._id: 64196e45060a651c415d5cf7
paper_authors.avatarUrl: /avatars/71a43232a7bae851eca252782387a63d.svg
paper_authors.fullname: Shiduo Zhang
paper_authors.type: user
paper_authors.user: CyberDJ
paper_authors._id: 67d38a42d3d16e1166d81bf1
paper_authors.name: Panpan Cai
paper_authors._id: 67d38a42d3d16e1166d81bf2
paper_authors.name: Jinlan Fu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:05:38.981Z
paper_authors._id: 618497ea8aaadc9253c2dfa9
paper_authors.avatarUrl: /avatars/2eb3954a99f5aede6f31b8ae49b8c910.svg
paper_authors.fullname: Fu Jinlan
paper_authors.type: user
paper_authors.user: Jinlan
paper_authors._id: 67d38a42d3d16e1166d81bf3
paper_authors.name: Xipeng Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:05:46.041Z
paper_authors._id: 61457b8deff2c9fdb4de4988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
paper_authors.fullname: Xipeng Qiu
paper_authors.type: user
paper_authors.user: xpqiu
paper_title: World Modeling Makes a Better Planner: Dual Preference Optimization for
    Embodied Task Planning
paper_summary: Recent advances in large vision-language models (LVLMs) have shown promise
  for embodied task planning, yet they struggle with fundamental challenges like
  dependency constraints and efficiency. Existing approaches either solely
  optimize action selection or leverage world models during inference,
  overlooking the benefits of learning to model the world as a way to enhance
  planning capabilities. We propose Dual Preference Optimization (D^2PO), a new
  learning framework that jointly optimizes state prediction and action selection
  through preference learning, enabling LVLMs to understand environment dynamics
  for better planning. To automatically collect trajectories and stepwise
  preference data without human annotation, we introduce a tree search mechanism
  for extensive exploration via trial-and-error. Extensive experiments on
  VoTa-Bench demonstrate that our D^2PO-based method significantly outperforms
  existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and
  LLaMA-3.2 (11B), achieving superior task success rates with more efficient
  execution paths.
paper_ai_keywords: Dual Preference Optimization (D$^2$PO) preference learning state prediction action selection environment dynamics tree search mechanism VoTa-Bench Qwen2-VL LLaVA-1.6 LLaMA-3.2 task success rates efficient execution paths