10

Figure 6: Success rate by memory size for different items. We
evaluated the performance of JARVIS-1 at different memory sizes
(representing different learning stages) by measuring the success
rate (% Episodes) of completing key items on the Minecraft tech-
nology tree. As the learning progressed, we observed an improve-
ment in completion rates for all items, with an increasing number
of successful trajectories being included in memory. After 4 epochs
of learning, JARVIS-1 had accumulated a total of 425 successful
trajectories in its memory.

Table 3: Success rates for different LLMs on Minecraft tasks.
PT and FT represent pre-trained and fine-tuned LLaMA2.

Task GPT-4 ChatGPT LLaMA2 PT LLaMA2 FT

0.97
0.96
0.94
0.34
0.09

0.95
0.95
0.90
0.30
0.05

0.55
0.50
0.35
0.05
0.00

0.85
0.85
0.75
0.25
0.05
JARVIS-1