RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning
  via Generative Simulation
We present RoboGen, a generative robotic agent that automatically learns
diverse robotic skills at scale via generative simulation. RoboGen leverages
the latest advancements in foundation and generative models. Instead of
directly using or adapting these models to produce policies or low-level
actions, we advocate for a generative scheme, which uses these models to
automatically generate diversified tasks, scenes, and training supervisions,
thereby scaling up robotic skill learning with minimal human supervision. Our
approach equips a robotic agent with a self-guided propose-generate-learn
cycle: the agent first proposes interesting tasks and skills to develop, and
then generates corresponding simulation environments by populating pertinent
objects and assets with proper spatial configurations. Afterwards, the agent
decomposes the proposed high-level task into sub-tasks, selects the optimal
learning approach (reinforcement learning, motion planning, or trajectory
optimization), generates required training supervision, and then learns
policies to acquire the proposed skill. Our work attempts to extract the
extensive and versatile knowledge embedded in large-scale models and transfer
them to the field of robotics. Our fully generative pipeline can be queried
repeatedly, producing an endless stream of skill demonstrations associated with
diverse tasks and environments.
6545009d6f006ce587180c81
Yufei Wang
admin_assigned
2023-11-03T15:56:06.992Z
64a48f758b961fd6fa385737
/avatars/328fdf1a4dca5f2bafbedf1d4af83f7c.svg
Yufei Wang
user
lomo-1
6545009d6f006ce587180c82
Zhou Xian
claimed_verified
2023-11-04T10:17:36.650Z
654533b7407bb19ff5335178
/avatars/043522ed4a481c74b0074f169eb7196f.svg
Zhou Xian
user
zhouxian
6545009d6f006ce587180c83
Feng Chen
6545009d6f006ce587180c84
Tsun-Hsuan Wang
6545009d6f006ce587180c85
Yian Wang
admin_assigned
2023-11-03T15:57:43.290Z
63897a90ec1f539adc0eeb23
/avatars/7094ccda0b88d47e286eb0f66c5632ed.svg
Yian Wang
user
jazzwang
6545009d6f006ce587180c86
Katerina Fragkiadaki
6545009d6f006ce587180c87
Zackory Erickson
6545009d6f006ce587180c88
David Held
6545009d6f006ce587180c89
Chuang Gan
RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning
  via Generative Simulation
We present RoboGen, a generative robotic agent that automatically learns
diverse robotic skills at scale via generative simulation. RoboGen leverages
the latest advancements in foundation and generative models. Instead of
directly using or adapting these models to produce policies or low-level
actions, we advocate for a generative scheme, which uses these models to
automatically generate diversified tasks, scenes, and training supervisions,
thereby scaling up robotic skill learning with minimal human supervision. Our
approach equips a robotic agent with a self-guided propose-generate-learn
cycle: the agent first proposes interesting tasks and skills to develop, and
then generates corresponding simulation environments by populating pertinent
objects and assets with proper spatial configurations. Afterwards, the agent
decomposes the proposed high-level task into sub-tasks, selects the optimal
learning approach (reinforcement learning, motion planning, or trajectory
optimization), generates required training supervision, and then learns
policies to acquire the proposed skill. Our work attempts to extract the
extensive and versatile knowledge embedded in large-scale models and transfer
them to the field of robotics. Our fully generative pipeline can be queried
repeatedly, producing an endless stream of skill demonstrations associated with
diverse tasks and environments.
RoboGen automates the learning of diverse robotic skills at scale using generative simulation to propose tasks and generate training environments and supervision.
generative robotic agent generative simulation foundation models generative models propose-generate-learn cycle reinforcement learning motion planning trajectory optimization