  We propose a novel approach to localize a 3D object from the intensity and
depth information images provided by a Time-of-Flight (ToF) sensor. Our method
uses two CNNs. The first one uses raw depth and intensity images as input, to
segment the floor pixels, from which the extrinsic parameters of the camera are
estimated. The second CNN is in charge of segmenting the object-of-interest. As
a main innovation, it exploits the calibration estimated from the prediction of
the first CNN to represent the geometric depth information in a coordinate
system that is attached to the ground, and is thus independent of the camera
elevation. In practice, both the height of pixels with respect to the ground,
and the orientation of normals to the point cloud are provided as input to the
second CNN. Given the segmentation predicted by the second CNN, the object is
localized based on point cloud alignment with a reference model. Our
experiments demonstrate that our proposed two-step approach improves
segmentation and localization accuracy by a significant margin compared to a
conventional CNN architecture, ignoring calibration and height maps, but also
compared to PointNet++.

Computer Vision and Pattern Recognition