title: RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
summary: Recent advancements in Artificial Intelligence (AI) have largely been
  propelled by scaling. In Robotics, scaling is hindered by the lack of access to
  massive robot datasets. We advocate using realistic physical simulation as a
  means to scale environments, tasks, and datasets for robot learning methods. We
  present RoboCasa, a large-scale simulation framework for training generalist
  robots in everyday environments. RoboCasa features realistic and diverse scenes
  focusing on kitchen environments. We provide thousands of 3D assets across over
  150 object categories and dozens of interactable furniture and appliances. We
  enrich the realism and diversity of our simulation with generative AI tools,
  such as object assets from text-to-3D models and environment textures from
  text-to-image models. We design a set of 100 tasks for systematic evaluation,
  including composite tasks generated by the guidance of large language models.
  To facilitate learning, we provide high-quality human demonstrations and
  integrate automated trajectory generation methods to substantially enlarge our
  datasets with minimal human burden. Our experiments show a clear scaling trend
  in using synthetically generated robot data for large-scale imitation learning
  and show great promise in harnessing simulation data in real-world tasks.
  Videos and open-source code are available at https://robocasa.ai/
paper_authors._id: 665fe6f5b4fb1c0cb09d12ed
paper_authors.name: Soroush Nasiriany
paper_authors._id: 665fe6f5b4fb1c0cb09d12ee
paper_authors.name: Abhiram Maddukuri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:39:36.510Z
paper_authors._id: 63a753248729ce5b5642e096
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1671910128497-noauth.jpeg
paper_authors.fullname: Abhiram Maddukuri
paper_authors.type: user
paper_authors.user: Abhi03
paper_authors._id: 665fe6f5b4fb1c0cb09d12ef
paper_authors.name: Lance Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:39:43.772Z
paper_authors._id: 64cce61fc176cd9dfa950b02
paper_authors.avatarUrl: /avatars/8b67572bf9ef271d7f5f5e46ff2b6fee.svg
paper_authors.fullname: Lance Zhang
paper_authors.type: user
paper_authors.user: LanceZhang
paper_authors._id: 665fe6f5b4fb1c0cb09d12f0
paper_authors.name: Adeet Parikh
paper_authors._id: 665fe6f5b4fb1c0cb09d12f1
paper_authors.name: Aaron Lo
paper_authors._id: 665fe6f5b4fb1c0cb09d12f2
paper_authors.name: Abhishek Joshi
paper_authors._id: 665fe6f5b4fb1c0cb09d12f3
paper_authors.name: Ajay Mandlekar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:40:25.049Z
paper_authors._id: 653ab89a7c01c693a17d88f7
paper_authors.avatarUrl: /avatars/30f3428d5cd04b2926b80c806126d827.svg
paper_authors.fullname: Ajay Mandlekar
paper_authors.type: user
paper_authors.user: amandlek
paper_authors._id: 665fe6f5b4fb1c0cb09d12f4
paper_authors.name: Yuke Zhu
paper_title: RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
paper_summary: Recent advancements in Artificial Intelligence (AI) have largely been
  propelled by scaling. In Robotics, scaling is hindered by the lack of access to
  massive robot datasets. We advocate using realistic physical simulation as a
  means to scale environments, tasks, and datasets for robot learning methods. We
  present RoboCasa, a large-scale simulation framework for training generalist
  robots in everyday environments. RoboCasa features realistic and diverse scenes
  focusing on kitchen environments. We provide thousands of 3D assets across over
  150 object categories and dozens of interactable furniture and appliances. We
  enrich the realism and diversity of our simulation with generative AI tools,
  such as object assets from text-to-3D models and environment textures from
  text-to-image models. We design a set of 100 tasks for systematic evaluation,
  including composite tasks generated by the guidance of large language models.
  To facilitate learning, we provide high-quality human demonstrations and
  integrate automated trajectory generation methods to substantially enlarge our
  datasets with minimal human burden. Our experiments show a clear scaling trend
  in using synthetically generated robot data for large-scale imitation learning
  and show great promise in harnessing simulation data in real-world tasks.
  Videos and open-source code are available at https://robocasa.ai/
paper_ai_keywords: physical simulation robot learning methods generalist robots kitchen environments 3D assets interactable furniture interactable appliances generative AI tools text-to-3D models text-to-image models large language models high-quality human demonstrations automated trajectory generation imitation learning synthetic generated robot data