  Semantic scene completion (SSC) is a challenging Computer Vision task with
many practical applications, from robotics to assistive computing. Its goal is
to infer the 3D geometry in a field of view of a scene and the semantic labels
of voxels, including occluded regions. In this work, we present SPAwN, a novel
lightweight multimodal 3D deep CNN that seamlessly fuses structural data from
the depth component of RGB-D images with semantic priors from a bimodal 2D
segmentation network. A crucial difficulty in this field is the lack of fully
labeled real-world 3D datasets which are large enough to train the current
data-hungry deep 3D CNNs. In 2D computer vision tasks, many data augmentation
strategies have been proposed to improve the generalization ability of CNNs.
However those approaches cannot be directly applied to the RGB-D input and
output volume of SSC solutions. In this paper, we introduce the use of a 3D
data augmentation strategy that can be applied to multimodal SSC networks. We
validate our contributions with a comprehensive and reproducible ablation
study. Our solution consistently surpasses previous works with a similar level
of complexity.

Computer Vision and Pattern Recognition;Artificial Intelligence