Mesh-based Object Tracking for Dynamic Semantic 3D Scene Graphs via Ray Tracing
<|reference_start|>Mesh-based Object Tracking for Dynamic Semantic 3D Scene Graphs via Ray Tracing: In this paper, we present a novel method for 3D geometric scene graph generation using range sensors and RGB cameras. We initially detect instance-wise keypoints with a YOLOv8s model to compute 6D pose estimates of known objects by solving PnP. We use a ray tracing approach to track a geometric scene graph consisting of mesh models of object instances. In contrast to classical point-to-point matching, this leads to more robust results, especially under occlusions between objects instances. We show that using this hybrid strategy leads to robust self-localization, pre-segmentation of the range sensor data and accurate pose tracking of objects using the same environmental representation. All detected objects are integrated into a semantic scene graph. This scene graph then serves as a front end to a semantic mapping framework to allow spatial reasoning.<|reference_end|>
@article{niecksch2024mesh-based,
    title={Mesh-based Object Tracking for Dynamic Semantic 3D Scene Graphs via Ray
  Tracing},
    author={Lennart Niecksch, Alexander Mock, Felix Igelbrink, Thomas Wiemann,
  Joachim Hertzberg},
    journal={RSS Workshop on Semantics for Robotics: From Environment
  Understanding and Reasoning to Safe Interaction 2024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.04979},
    primaryClass={cs.RO}
}