title: RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning
    via Generative Simulation
summary: We present RoboGen, a generative robotic agent that automatically learns
  diverse robotic skills at scale via generative simulation. RoboGen leverages
  the latest advancements in foundation and generative models. Instead of
  directly using or adapting these models to produce policies or low-level
  actions, we advocate for a generative scheme, which uses these models to
  automatically generate diversified tasks, scenes, and training supervisions,
  thereby scaling up robotic skill learning with minimal human supervision. Our
  approach equips a robotic agent with a self-guided propose-generate-learn
  cycle: the agent first proposes interesting tasks and skills to develop, and
  then generates corresponding simulation environments by populating pertinent
  objects and assets with proper spatial configurations. Afterwards, the agent
  decomposes the proposed high-level task into sub-tasks, selects the optimal
  learning approach (reinforcement learning, motion planning, or trajectory
  optimization), generates required training supervision, and then learns
  policies to acquire the proposed skill. Our work attempts to extract the
  extensive and versatile knowledge embedded in large-scale models and transfer
  them to the field of robotics. Our fully generative pipeline can be queried
  repeatedly, producing an endless stream of skill demonstrations associated with
  diverse tasks and environments.
paper_authors._id: 6545009d6f006ce587180c81
paper_authors.name: Yufei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-03T15:56:06.992Z
paper_authors._id: 64a48f758b961fd6fa385737
paper_authors.avatarUrl: /avatars/328fdf1a4dca5f2bafbedf1d4af83f7c.svg
paper_authors.fullname: Yufei Wang
paper_authors.type: user
paper_authors.user: lomo-1
paper_authors._id: 6545009d6f006ce587180c82
paper_authors.name: Zhou Xian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-04T10:17:36.650Z
paper_authors._id: 654533b7407bb19ff5335178
paper_authors.avatarUrl: /avatars/043522ed4a481c74b0074f169eb7196f.svg
paper_authors.fullname: Zhou Xian
paper_authors.type: user
paper_authors.user: zhouxian
paper_authors._id: 6545009d6f006ce587180c83
paper_authors.name: Feng Chen
paper_authors._id: 6545009d6f006ce587180c84
paper_authors.name: Tsun-Hsuan Wang
paper_authors._id: 6545009d6f006ce587180c85
paper_authors.name: Yian Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-03T15:57:43.290Z
paper_authors._id: 63897a90ec1f539adc0eeb23
paper_authors.avatarUrl: /avatars/7094ccda0b88d47e286eb0f66c5632ed.svg
paper_authors.fullname: Yian Wang
paper_authors.type: user
paper_authors.user: jazzwang
paper_authors._id: 6545009d6f006ce587180c86
paper_authors.name: Katerina Fragkiadaki
paper_authors._id: 6545009d6f006ce587180c87
paper_authors.name: Zackory Erickson
paper_authors._id: 6545009d6f006ce587180c88
paper_authors.name: David Held
paper_authors._id: 6545009d6f006ce587180c89
paper_authors.name: Chuang Gan
paper_title: RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning
    via Generative Simulation
paper_summary: We present RoboGen, a generative robotic agent that automatically learns
  diverse robotic skills at scale via generative simulation. RoboGen leverages
  the latest advancements in foundation and generative models. Instead of
  directly using or adapting these models to produce policies or low-level
  actions, we advocate for a generative scheme, which uses these models to
  automatically generate diversified tasks, scenes, and training supervisions,
  thereby scaling up robotic skill learning with minimal human supervision. Our
  approach equips a robotic agent with a self-guided propose-generate-learn
  cycle: the agent first proposes interesting tasks and skills to develop, and
  then generates corresponding simulation environments by populating pertinent
  objects and assets with proper spatial configurations. Afterwards, the agent
  decomposes the proposed high-level task into sub-tasks, selects the optimal
  learning approach (reinforcement learning, motion planning, or trajectory
  optimization), generates required training supervision, and then learns
  policies to acquire the proposed skill. Our work attempts to extract the
  extensive and versatile knowledge embedded in large-scale models and transfer
  them to the field of robotics. Our fully generative pipeline can be queried
  repeatedly, producing an endless stream of skill demonstrations associated with
  diverse tasks and environments.
paper_ai_keywords: generative simulation foundation models generative models propose-generate-learn cycle high-level task decomposition sub-tasks reinforcement learning motion planning trajectory optimization training supervision policy learning skill demonstrations diverse tasks diverse environments extensive knowledge transfer large-scale models