title: ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding
summary: The performance of neural networks scales with both their size and the amount
  of data they have been trained on. This is shown in both language and image
  generation. However, this requires scaling-friendly network architectures as
  well as large-scale datasets. Even though scaling-friendly architectures like
  transformers have emerged for 3D vision tasks, the GPT-moment of 3D vision
  remains distant due to the lack of training data. In this paper, we introduce
  ARKit LabelMaker, the first large-scale, real-world 3D dataset with dense
  semantic annotations. Specifically, we complement ARKitScenes dataset with
  dense semantic annotations that are automatically generated at scale. To this
  end, we extend LabelMaker, a recent automatic annotation pipeline, to serve the
  needs of large-scale pre-training. This involves extending the pipeline with
  cutting-edge segmentation models as well as making it robust to the challenges
  of large-scale processing. Further, we push forward the state-of-the-art
  performance on ScanNet and ScanNet200 dataset with prevalent 3D semantic
  segmentation models, demonstrating the efficacy of our generated dataset.
paper_authors._id: 6717f63e80594ffee6df3448
paper_authors.name: Guangda Ji
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-24T12:34:35.954Z
paper_authors._id: 62c7368a6a092eda1f1f8509
paper_authors.avatarUrl: /avatars/b18d5fe7491e97d9eabe26051cfe61a3.svg
paper_authors.fullname: Guangda Ji
paper_authors.type: user
paper_authors.user: quantaji
paper_authors._id: 6717f63e80594ffee6df3449
paper_authors.name: Silvan Weder
paper_authors._id: 6717f63e80594ffee6df344a
paper_authors.name: Francis Engelmann
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:31:05.243Z
paper_authors._id: 671a7160e45d92e304e82f45
paper_authors.avatarUrl: /avatars/2b0bf3b27929cbf23e99a3d432d1e9f7.svg
paper_authors.fullname: Francis Engelmann
paper_authors.type: user
paper_authors.user: francisengelmann
paper_authors._id: 6717f63e80594ffee6df344b
paper_authors.name: Marc Pollefeys
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:41:40.107Z
paper_authors._id: 67b5fa179782a5e2fd2cb26a
paper_authors.avatarUrl: /avatars/62c38f29ec641e001eeddf840bea21a0.svg
paper_authors.fullname: Marc Pollefeys
paper_authors.type: user
paper_authors.user: mapo1
paper_authors._id: 6717f63e80594ffee6df344c
paper_authors.name: Hermann Blum
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-24T12:35:12.197Z
paper_authors._id: 63ff8abfb09f82a81a1e545f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/e7lJKlza9ydEAXJYI9Bx6.png
paper_authors.fullname: Hermann Blum
paper_authors.type: user
paper_authors.user: blumh
paper_title: ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding
paper_summary: The performance of neural networks scales with both their size and the amount
  of data they have been trained on. This is shown in both language and image
  generation. However, this requires scaling-friendly network architectures as
  well as large-scale datasets. Even though scaling-friendly architectures like
  transformers have emerged for 3D vision tasks, the GPT-moment of 3D vision
  remains distant due to the lack of training data. In this paper, we introduce
  ARKit LabelMaker, the first large-scale, real-world 3D dataset with dense
  semantic annotations. Specifically, we complement ARKitScenes dataset with
  dense semantic annotations that are automatically generated at scale. To this
  end, we extend LabelMaker, a recent automatic annotation pipeline, to serve the
  needs of large-scale pre-training. This involves extending the pipeline with
  cutting-edge segmentation models as well as making it robust to the challenges
  of large-scale processing. Further, we push forward the state-of-the-art
  performance on ScanNet and ScanNet200 dataset with prevalent 3D semantic
  segmentation models, demonstrating the efficacy of our generated dataset.
paper_ai_keywords: transformers ARKit LabelMaker ARKit Scenes automatic annotation pipeline segmentation models ScanNet ScanNet200 3D semantic segmentation large-scale pre-training dense semantic annotations