within the game. Hallucinations also occur during the code generation process. For instance, GPT-4
tends to use cobblestone as a fuel input, despite being an invalid fuel source in the game. Additionally,
it may call functions absent in the provided control primitive APIs, leading to code execution errors.
We are confident that improvements in the GPT API models as well as novel techniques for finetuning
open-source LLMs will overcome these limitations in the future.
VOYAGER- An Open-Ended Embodied Agent with Large Language Models