Mono SF: Multi View Geometry Meets Single View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes
Existing 3D scene flow estimation methods provide the 3D geometry and 3D motion of a scene and gain a lot of interest, for example in the context of autonomous driving. These methods are traditionally based on a temporal series of stereo images. In this paper, we propose a novel monocular 3D scene flow estimation method, called Mono SF. Mono SF jointly estimates the 3D structure and motion of the scene by combining multi view geometry and single view depth information. Mono SF considers that the scene flow should be consistent in terms of warping the reference image in the consecutive image based on the principles of multi view geometry. For integrating single view depth in a statistical manner, a convolutional neural network, called ProbDepthNet, is proposed. ProbDepthNet estimates pixel wise depth distributions from a single image rather than single depth values. Additionally, as part of ProbDepthNet, a novel recalibration technique for regression problems is proposed to ensure well calibrated distributions. Our experiments show that Mono SF outperforms state of the art monocular baselines and ablation studies support the Mono SF approach and ProbDepthNet design.
2019 IEEE/CVF International Conference on Computer Vision (ICCV)