3D Object Proposals for Accurate Object Class Detection
The goal of this paper is to generate high-quality 3D object proposals in the context of autonomous driving. Our method exploits stereo imagery to place proposals in the form of 3D bounding boxes. We formulate the problem as minimizing an energy function encoding object size priors, ground plane as well as several depth informed features that reason about free space, point cloud densities and distance to the ground. Our experiments show significant performance gains over existing RGB and RGB-D object proposal methods on the challenging KITTI benchmark. Combined with convolutional neural net (CNN) scoring, our approach outperforms all existing results on all three KITTI object classes.
-- Summary --This paper addresses the problem of generating 3D object proposals given a stereo image pair from an autonomous driving vehicle. The paper proposes a set of features for a 3D cuboid over a point cloud and ground plane derived from the stereo image pair. The features include point cloud density, free space, object height prior, and object height relative to its surroundings. Note that the features are dependant on knowledge of the object class (other "objectness" proposal methods are agnostic to the object class). A structural SVM is trained to predict the "objectness" of the 3D cuboid proposal. Bounding box and orientation regression is performed for the final object detection.The approach is evaluated on the KITTI benchmark, and the approach outperforms competing baselines for oracle recall and object detection. -- Comments --Positives: Generating good object proposals has shown to be an important step for object detection.While most prior work has focused on 2D still images, this paper focuses on 3D data, which has arguably received less attention. The approach outperforms baselines in this space (including approaches informed by 3D), which also translates to improved performance for object detection.The paper is reasonably clear to get the ideas of the approach and cites well prior art in this space.Negatives: The main ideas behind the paper may be perceived as incremental from a machine learning perspective.I think the boost in performance would be appreciated more by the 3D vision community.Detailed comments:+ Section 3.4: I'm not clear why a structured SVM is used here. Section 3.1 introduced a feature vector, and it seems that only a linear classifier is needed.I'd like to see how performance of a linear SVM with hard negative mining would perform.+ Prior work in generating object proposals are object class agnostic.I'd like to see performance where object class information is not used (e.g. remove object height from feature vector).  + Line 107: It would be good to clarify that the input is a calibrated stereo image pair since the approach relies on metric information. Also perhaps mention the dependency on ground plane estimation.+ Paragraph starting Line 142: What is the parameterization of t? The indices for t and \theta are not needed.Finally, it may be good to mention that it is assumed the boxes live on the ground and/or its base is parallel to the ground plane.+ Equation (1): It may be good to make explicit that each \phi depends on x.+ Equation (4): Why not "max" over the heights?+ Line 194: I think this potential assumes objects are well segmented in 3D (i.e. no overlap or intersection).Is this a valid assumption?+ Line 246: Please provide more details on the ground plane estimation, as it is not reproducible as described.+ Line 255: NMS => non maximum suppression+ Section 3.4: What is the setting of C?Also, is "y" supposed to be bold?