Voyager: An Open-Ended Embodied Agent with Large Language Models
We introduce Voyager, the first LLM-powered embodied lifelong learning agent
in Minecraft that continuously explores the world, acquires diverse skills, and
makes novel discoveries without human intervention. Voyager consists of three
key components: 1) an automatic curriculum that maximizes exploration, 2) an
ever-growing skill library of executable code for storing and retrieving
complex behaviors, and 3) a new iterative prompting mechanism that incorporates
environment feedback, execution errors, and self-verification for program
improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses
the need for model parameter fine-tuning. The skills developed by Voyager are
temporally extended, interpretable, and compositional, which compounds the
agent's abilities rapidly and alleviates catastrophic forgetting. Empirically,
Voyager shows strong in-context lifelong learning capability and exhibits
exceptional proficiency in playing Minecraft. It obtains 3.3x more unique
items, travels 2.3x longer distances, and unlocks key tech tree milestones up
to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill
library in a new Minecraft world to solve novel tasks from scratch, while other
techniques struggle to generalize. We open-source our full codebase and prompts
at https://voyager.minedojo.org/.
64700d67644111130400caaf
Guanzhi Wang
claimed_verified
2023-05-26T18:35:19.784Z
628353d76db5127097ce65cd
/avatars/b8e519114d0926db4e2f9a0514e5cb32.svg
Guanzhi Wang
user
guanzhi
64700d67644111130400cab0
Yuqi Xie
claimed_verified
2023-05-26T13:57:22.122Z
63f3d5f04745321de355fb26
https://cdn-avatars.huggingface.co/v1/production/uploads/1676924371675-noauth.png
Yuqi Xie
user
xieleo5
64700d67644111130400cab1
Yunfan Jiang
claimed_verified
2023-05-26T17:15:03.477Z
61e9f5398c237a147a3f4ab5
/avatars/afd4ec17cb132b5ab56e50a678c4786d.svg
Yunfan Jiang
user
yunfanj
64700d67644111130400cab2
Ajay Mandlekar
64700d67644111130400cab3
Chaowei Xiao
64700d67644111130400cab4
Yuke Zhu
64700d67644111130400cab5
Linxi Fan
claimed_verified
2023-05-28T07:08:29.114Z
628352f309aa80237c6c3e4b
https://cdn-avatars.huggingface.co/v1/production/uploads/628352f309aa80237c6c3e4b/W2dkeN-jJ137HfbYKo088.jpeg
Jim Fan
user
jimfan
64700d67644111130400cab6
Anima Anandkumar
Voyager: An Open-Ended Embodied Agent with Large Language Models
We introduce Voyager, the first LLM-powered embodied lifelong learning agent
in Minecraft that continuously explores the world, acquires diverse skills, and
makes novel discoveries without human intervention. Voyager consists of three
key components: 1) an automatic curriculum that maximizes exploration, 2) an
ever-growing skill library of executable code for storing and retrieving
complex behaviors, and 3) a new iterative prompting mechanism that incorporates
environment feedback, execution errors, and self-verification for program
improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses
the need for model parameter fine-tuning. The skills developed by Voyager are
temporally extended, interpretable, and compositional, which compounds the
agent's abilities rapidly and alleviates catastrophic forgetting. Empirically,
Voyager shows strong in-context lifelong learning capability and exhibits
exceptional proficiency in playing Minecraft. It obtains 3.3x more unique
items, travels 2.3x longer distances, and unlocks key tech tree milestones up
to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill
library in a new Minecraft world to solve novel tasks from scratch, while other
techniques struggle to generalize. We open-source our full codebase and prompts
at https://voyager.minedojo.org/.
Voyager is an LLM-powered agent in Minecraft that autonomously explores, learns, and discovers skills through a curriculum, skill library, and prompting mechanism, demonstrating superior lifelong learning and task-solving capabilities.
LLM-powered embodied lifelong learning agent automatic curriculum skill library iterative prompting mechanism environment feedback execution errors self-verification blackbox queries GPT-4 temporal extension interpretable compositional catastrophic forgetting in-context lifelong learning unique items distances tech tree milestones novel tasks generalization Minedojo