title: Text2World: Benchmarking Large Language Models for Symbolic World Model
    Generation
summary: Recently, there has been growing interest in leveraging large language models
  (LLMs) to generate symbolic world models from textual descriptions. Although
  LLMs have been extensively explored in the context of world modeling, prior
  studies encountered several challenges, including evaluation randomness,
  dependence on indirect metrics, and a limited domain scope. To address these
  limitations, we introduce a novel benchmark, Text2World, based on planning
  domain definition language (PDDL), featuring hundreds of diverse domains and
  employing multi-criteria, execution-based metrics for a more robust evaluation.
  We benchmark current LLMs using Text2World and find that reasoning models
  trained with large-scale reinforcement learning outperform others. However,
  even the best-performing model still demonstrates limited capabilities in world
  modeling. Building on these insights, we examine several promising strategies
  to enhance the world modeling capabilities of LLMs, including test-time
  scaling, agent training, and more. We hope that Text2World can serve as a
  crucial resource, laying the groundwork for future research in leveraging LLMs
  as world models. The project page is available at
  https://text-to-world.github.io/.
paper_authors._id: 67b5473109afe1f3029835cb
paper_authors.name: Mengkang Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T09:01:15.592Z
paper_authors._id: 6237df4a5ab9df625fb70c1a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6237df4a5ab9df625fb70c1a/rmtSgp-_WvHZBXnzPrXL0.jpeg
paper_authors.fullname: Mengkang Hu
paper_authors.type: user
paper_authors.user: MengkangHu
paper_authors._id: 67b5473109afe1f3029835cc
paper_authors.name: Tianxing Chen
paper_authors._id: 67b5473109afe1f3029835cd
paper_authors.name: Yude Zou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:37:22.709Z
paper_authors._id: 67b4123c9a692ad23daa8a73
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67b4123c9a692ad23daa8a73/vhkxv0NkM2socQ2tySzqc.jpeg
paper_authors.fullname: Yude Zou
paper_authors.type: user
paper_authors.user: xdzouyd
paper_authors._id: 67b5473109afe1f3029835ce
paper_authors.name: Yuheng Lei
paper_authors._id: 67b5473109afe1f3029835cf
paper_authors.name: Qiguang Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T09:01:12.095Z
paper_authors._id: 636f526a6cd69d9a36ff2b53
paper_authors.avatarUrl: /avatars/8f2271a193fcac609d9be270552b5afa.svg
paper_authors.fullname: Qiguang Chen
paper_authors.type: user
paper_authors.user: LightChen2333
paper_authors._id: 67b5473109afe1f3029835d0
paper_authors.name: Ming Li
paper_authors._id: 67b5473109afe1f3029835d1
paper_authors.name: Hongyuan Zhang
paper_authors._id: 67b5473109afe1f3029835d2
paper_authors.name: Wenqi Shao
paper_authors._id: 67b5473109afe1f3029835d3
paper_authors.name: Ping Luo
paper_title: Text2World: Benchmarking Large Language Models for Symbolic World Model
    Generation
paper_summary: Recently, there has been growing interest in leveraging large language models
  (LLMs) to generate symbolic world models from textual descriptions. Although
  LLMs have been extensively explored in the context of world modeling, prior
  studies encountered several challenges, including evaluation randomness,
  dependence on indirect metrics, and a limited domain scope. To address these
  limitations, we introduce a novel benchmark, Text2World, based on planning
  domain definition language (PDDL), featuring hundreds of diverse domains and
  employing multi-criteria, execution-based metrics for a more robust evaluation.
  We benchmark current LLMs using Text2World and find that reasoning models
  trained with large-scale reinforcement learning outperform others. However,
  even the best-performing model still demonstrates limited capabilities in world
  modeling. Building on these insights, we examine several promising strategies
  to enhance the world modeling capabilities of LLMs, including test-time
  scaling, agent training, and more. We hope that Text2World can serve as a
  crucial resource, laying the groundwork for future research in leveraging LLMs
  as world models. The project page is available at
  https://text-to-world.github.io/.
paper_ai_summary: Text2World, a novel PDDL-based benchmark, evaluates world modeling capabilities of LLMs using diverse domains and multi-criteria metrics, highlighting the need for enhanced strategies.
paper_ai_keywords: large language models symbolic world models Text2World planning domain definition language reinforcement learning test-time scaling agent training