ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding
The performance of neural networks scales with both their size and the amount
of data they have been trained on. This is shown in both language and image
generation. However, this requires scaling-friendly network architectures as
well as large-scale datasets. Even though scaling-friendly architectures like
transformers have emerged for 3D vision tasks, the GPT-moment of 3D vision
remains distant due to the lack of training data. In this paper, we introduce
ARKit LabelMaker, the first large-scale, real-world 3D dataset with dense
semantic annotations. Specifically, we complement ARKitScenes dataset with
dense semantic annotations that are automatically generated at scale. To this
end, we extend LabelMaker, a recent automatic annotation pipeline, to serve the
needs of large-scale pre-training. This involves extending the pipeline with
cutting-edge segmentation models as well as making it robust to the challenges
of large-scale processing. Further, we push forward the state-of-the-art
performance on ScanNet and ScanNet200 dataset with prevalent 3D semantic
segmentation models, demonstrating the efficacy of our generated dataset.
6717f63e80594ffee6df3448
Guangda Ji
admin_assigned
2024-10-24T12:34:35.954Z
62c7368a6a092eda1f1f8509
/avatars/b18d5fe7491e97d9eabe26051cfe61a3.svg
Guangda Ji
user
quantaji
6717f63e80594ffee6df3449
Silvan Weder
6717f63e80594ffee6df344a
Francis Engelmann
claimed_verified
2024-10-25T09:31:05.243Z
671a7160e45d92e304e82f45
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-trHIg97qX0dE9T4UYycc.png
Francis Engelmann
user
francisengelmann
6717f63e80594ffee6df344b
Marc Pollefeys
claimed_verified
2025-02-25T09:41:40.107Z
67b5fa179782a5e2fd2cb26a
/avatars/62c38f29ec641e001eeddf840bea21a0.svg
Marc Pollefeys
user
mapo1
6717f63e80594ffee6df344c
Hermann Blum
admin_assigned
2024-10-24T12:35:12.197Z
63ff8abfb09f82a81a1e545f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/e7lJKlza9ydEAXJYI9Bx6.png
Hermann Blum
user
blumh
ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding
The performance of neural networks scales with both their size and the amount
of data they have been trained on. This is shown in both language and image
generation. However, this requires scaling-friendly network architectures as
well as large-scale datasets. Even though scaling-friendly architectures like
transformers have emerged for 3D vision tasks, the GPT-moment of 3D vision
remains distant due to the lack of training data. In this paper, we introduce
ARKit LabelMaker, the first large-scale, real-world 3D dataset with dense
semantic annotations. Specifically, we complement ARKitScenes dataset with
dense semantic annotations that are automatically generated at scale. To this
end, we extend LabelMaker, a recent automatic annotation pipeline, to serve the
needs of large-scale pre-training. This involves extending the pipeline with
cutting-edge segmentation models as well as making it robust to the challenges
of large-scale processing. Further, we push forward the state-of-the-art
performance on ScanNet and ScanNet200 dataset with prevalent 3D semantic
segmentation models, demonstrating the efficacy of our generated dataset.
A large-scale, real-world 3D dataset with dense semantic annotations is introduced and demonstrated to enhance state-of-the-art performance in 3D semantic segmentation.
neural networks GPT transformers 3D vision dataset semantic annotations LabelMaker segmentation models ScanNet ScanNet200 semantic segmentation