Three-dimensional (3D) reconstruction is a process of computing mathematical representation of a 3D surface of an object from two-dimensional (2D) projections of the object obtained from different points of view.
Generally, the 3D reconstruction is based on various optical systems used for image capturing. The optical system may be configured either as a depth camera (for example, an active depth sensor or stereo camera), or as a monocular camera.
The depth camera may capture depth maps of an object or environment in real time. Here, the depth maps may be captured immediately in case of the active depth sensor or after processing a pair of rectified stereo images in case of the stereo camera. Each pixel in the depth maps corresponds to a distance from one point of the object to the depth camera. However, when the depth camera is included in a device to capture the depth maps, many hardware resources related to a depth sensor is required, and thus hardware configuration of the device may be complex.
A silhouette-based 3D reconstruction approach may use monocular camera-based approaches. In the silhouette-based 3D reconstruction approach, silhouettes of an image are extracted and then a volume structure of the object is formed. According to the silhouette-based 3D reconstruction approach, a polygonal mesh with color texture is generated. However, in the silhouette-based 3D reconstruction approach, complicated computations related to extraction of silhouettes and fusion of volume data need to be performed in a device having low processing capability, such as a mobile device. Also, in the silhouette-based 3D reconstruction approach, concavities that are not seen in silhouette images are unable to be recovered.
An example of a method using monocular camera-based hardware includes a structure-from-motion (SfM)-based 3D reconstruction process. In the SfM-based 3D reconstruction process, trajectories of extracted feature points are used to reconstruct camera motion and 3D positions of an object. However, in the SfM-based 3D reconstruction process, since a 3D point cloud or polygonal mesh is reconstructed without texture, visual appearance of a reconstructed 3D model is not sufficient. Also, in the SfM-based 3D reconstruction process, since a large number of feature points need to be processed in order to increase accuracy, computational workloads may increase.
Since the 3D reconstruction approaches described above require many complex computations, it may be infeasible to implement the 3D reconstruction approaches on mobile platforms having limited process performances, such as a smart phones or tablets. Cloud-based data processing may be used to overcome the limited process performances of the mobile platforms, but in this case, the mobile platforms need to be network accessible all the time and may be time consuming compared to the 3D reconstruction approaches described above.
Thus, there is a need for techniques of 3D reconstruction of an object, which may be performed by using less time and suitable computational workloads, and also without use of additional hardware resources.
The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the present disclosure.