are not speciﬁcally built to demonstrate understanding. They are trained to generate sequences of
words that might be plausible to a human given a prior context. In the terms of Mahowald et al. [4]
LLMs have become amazingly proﬁcient at linguistic competence — knowing how to say things;
but they are not nearly as good at functional competence — knowing what to say. Speciﬁcally, they
can be (relatively) easily fooled by, for example, asking for the result of a straightforward arithmetic
problem that does not appear in their training corpus or by asking for the solution to a novel planning
problem that requires knowledge about how the world works.
LLM+P- Empowering Large Language Models with Optimal Planning Proficiency