  Estimating the layout of a room from a single-shot panoramic image is
important in virtual/augmented reality and furniture layout simulation. This
involves identifying three-dimensional (3D) geometry, such as the location of
corners and boundaries, and performing 3D reconstruction. However, occlusion is
a common issue that can negatively impact room layout estimation, and this has
not been thoroughly studied to date. It is possible to obtain 3D shape
information of rooms as drawings of buildings and coordinates of corners from
image datasets, thus we propose providing both 2D panoramic and 3D information
to a model to effectively deal with occlusion. However, simply feeding 3D
information to a model is not sufficient to utilize the shape information for
an occluded area. Therefore, we improve the model by introducing 3D
Intersection over Union (IoU) loss to effectively use 3D information. In some
cases, drawings are not available or the construction deviates from a drawing.
Considering such practical cases, we propose a method for distilling knowledge
from a model trained with both images and 3D information to a model that takes
only images as input. The proposed model, which is called Shape-Net, achieves
state-of-the-art (SOTA) performance on benchmark datasets. We also confirmed
its effectiveness in dealing with occlusion through significantly improved
accuracy on images with occlusion compared with existing models.

Computer Vision and Pattern Recognition