5.3 Results and Analysis

The results of applying LLM-AS-P and LLM+P across 7 domains are provided in Table 1.
Findings (LLM-AS-P):

1. We observe that though LLM-AS-P provides a plan in natural language for every problem,
most of these plans are not feasible. The main reason is that LLM-AS-P lacks the ability
to reason about preconditions. Moreover, adding context (e.g., LLM-AS-P (w/ context))
does not increase the success rate unless the new solution can be built from the example
solution as a template.

2. The TYREWORLD domain has the most performance boost from LLM-AS-P (w/o context)
(0% success) to LLM-AS-P (w/ context) (40%). This domain has a ﬁxed sequence of
actions for replacing each tire, and the problems only vary in the number of tires that have
to be replaced.
LLM+P- Empowering Large Language Models with Optimal Planning Proficiency