A technique of improving cognitive processing pipelines for object detection that fuse image segmentation data (for example, segmentation scores) with LiDAR points is provided. The disclosed technique is implemented by using an architecture that accepts point clouds and images as inputs and estimates directed 3D bounding boxes for all relevant object classes. In an embodiment, the method comprises the steps of: temporally matching points in a three-dimensional (3D) point cloud with an image by using a vehicle's one or more processors; generating semantic data for the image by using an image-based neural network; decorating, by using the one or more processors, points in the 3D point cloud with semantic data; and estimating a directed 3D bounding box for one or more objects by using a 3D object detector having the decorated points as an input.
3D SEQUENTIAL FUSION FOR 3D OBJECT DETECTION