title: WALL-E: World Alignment by Rule Learning Improves World Model-based LLM
    Agents
summary: Can large language models (LLMs) directly serve as powerful world models for
  model-based agents? While the gaps between the prior knowledge of LLMs and the
  specified environment's dynamics do exist, our study reveals that the gaps can
  be bridged by aligning an LLM with its deployed environment and such "world
  alignment" can be efficiently achieved by rule learning on LLMs. Given the rich
  prior knowledge of LLMs, only a few additional rules suffice to align LLM
  predictions with the specified environment dynamics. To this end, we propose a
  neurosymbolic approach to learn these rules gradient-free through LLMs, by
  inducing, updating, and pruning rules based on comparisons of agent-explored
  trajectories and world model predictions. The resulting world model is composed
  of the LLM and the learned rules. Our embodied LLM agent "WALL-E" is built upon
  model-predictive control (MPC). By optimizing look-ahead actions based on the
  precise world model, MPC significantly improves exploration and learning
  efficiency. Compared to existing LLM agents, WALL-E's reasoning only requires a
  few principal rules rather than verbose buffered trajectories being included in
  the LLM input. On open-world challenges in Minecraft and ALFWorld, WALL-E
  achieves higher success rates than existing methods, with lower costs on
  replanning time and the number of tokens used for reasoning. In Minecraft,
  WALL-E exceeds baselines by 15-30% in success rate while costing 8-20 fewer
  replanning rounds and only 60-80% of tokens. In ALFWorld, its success rate
  surges to a new record high of 95% only after 6 iterations.
paper_authors._id: 670906442f30a8b4755bfd36
paper_authors.name: Siyu Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T11:08:51.027Z
paper_authors._id: 6588f504d2ea3f329443e768
paper_authors.avatarUrl: /avatars/ab8f3bc0844b02c5e26a450bc3ee7689.svg
paper_authors.fullname: Siyu Zhou
paper_authors.type: user
paper_authors.user: Sawyer99
paper_authors._id: 670906442f30a8b4755bfd37
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:16:12.525Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_authors._id: 670906442f30a8b4755bfd38
paper_authors.name: Yijun Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:16:09.783Z
paper_authors._id: 645e553c3b6d85c65e8b0e54
paper_authors.avatarUrl: /avatars/1fffc6499b9d65b21a895ca96f03b781.svg
paper_authors.fullname: Steven
paper_authors.type: user
paper_authors.user: yijunyang
paper_authors._id: 670906442f30a8b4755bfd39
paper_authors.name: Guodong Long
paper_authors._id: 670906442f30a8b4755bfd3a
paper_authors.name: Deheng Ye
paper_authors._id: 670906442f30a8b4755bfd3b
paper_authors.name: Jing Jiang
paper_authors._id: 670906442f30a8b4755bfd3c
paper_authors.name: Chengqi Zhang
paper_title: WALL-E: World Alignment by Rule Learning Improves World Model-based LLM
    Agents
paper_summary: Can large language models (LLMs) directly serve as powerful world models for
  model-based agents? While the gaps between the prior knowledge of LLMs and the
  specified environment's dynamics do exist, our study reveals that the gaps can
  be bridged by aligning an LLM with its deployed environment and such "world
  alignment" can be efficiently achieved by rule learning on LLMs. Given the rich
  prior knowledge of LLMs, only a few additional rules suffice to align LLM
  predictions with the specified environment dynamics. To this end, we propose a
  neurosymbolic approach to learn these rules gradient-free through LLMs, by
  inducing, updating, and pruning rules based on comparisons of agent-explored
  trajectories and world model predictions. The resulting world model is composed
  of the LLM and the learned rules. Our embodied LLM agent "WALL-E" is built upon
  model-predictive control (MPC). By optimizing look-ahead actions based on the
  precise world model, MPC significantly improves exploration and learning
  efficiency. Compared to existing LLM agents, WALL-E's reasoning only requires a
  few principal rules rather than verbose buffered trajectories being included in
  the LLM input. On open-world challenges in Minecraft and ALFWorld, WALL-E
  achieves higher success rates than existing methods, with lower costs on
  replanning time and the number of tokens used for reasoning. In Minecraft,
  WALL-E exceeds baselines by 15-30% in success rate while costing 8-20 fewer
  replanning rounds and only 60-80% of tokens. In ALFWorld, its success rate
  surges to a new record high of 95% only after 6 iterations.
paper_ai_keywords: Large language models (LLMs) World models Model-based agents Rule learning Neurosymbolic approach Gradient-free learning Inducing rules Updating rules Pruning rules Agent-explored trajectories World model predictions Model-predictive control (MPC) Precise world model Look-ahead actions Exploration efficiency Learning efficiency Verbose buffered trajectories Success rates Replanning time Tokens used for reasoning Open-world challenges Minecraft ALFWorld Baselines Replanning rounds Iterations