A recording device that generates an omnidirectional image obtained by mapping an image of a 360° range in a horizontal direction and an image of a 180° range in a vertical direction to a 2D image (a planar image) from captured images in all directions captured by multiple cameras and encodes and records the omnidirectional image is known (for example, see Patent Literature 1.
In such a recording device, as a method of generating the omnidirectional image, a method of performing mapping from a 3D image to a 2D image such as equidistant cylindrical projection or cube mapping is used. When the equidistant cylindrical projection is used as the method of generating the omnidirectional image, the omnidirectional image is an image obtained by performing the equidistant cylindrical projection on a sphere in which a captured image is mapped to a surface of the sphere. Further, when the cube mapping is used as the method of generating the omnidirectional image, the omnidirectional image is an image of a development view of a cube in which a captured image is mapped to a surface of the cube.
On the other hand, a reproducing device that decodes an encoded stream of an omnidirectional image recorded by a recording device and displays an image of a field-of-view range of a user using the omnidirectional image obtained as a result is known. Such a reproducing device attaches the omnidirectional image to a surface of a 3D model such as a sphere or a cube as a texture and displays the image of the field-of-view range of the user when the texture of the surface of the 3D model in a direction of a line-of-sight vector of the user is viewed from one point in the 3D model. Accordingly, the captured image of the field-of-view range of the user is reproduced.
This reproduction method is referred to as an “environmental light model” in the field of computer graphics (CG). Further, the environment light model of attaching an image generated by the equidistant cylindrical projection to the surface of the sphere is referred to as a “sphere model,” and the environment light model of attaching an image generated by the cube mapping to the surface of the cube surface is referred to as a “cube model.”
The captured image of the field-of-view range of the user can be reproduced using only an area corresponding to the line-of-sight vector of the user in the omnidirectional image. For example, when a viewing angle in the horizontal direction is 90°, in the horizontal direction, only ¼ of the omnidirectional image to which the images in all directions in a 360° range in the horizontal direction are mapped is used for reproduction of the captured image of the field-of-view range of the user. The sample applies to the vertical direction.
Therefore, it is wasteful to decode the encoded streams of the entire omnidirectional image. However, since the line-of-sight vector is decided by the user, the recording device is unable to encode only the area corresponding to the line-of-sight vector in the omnidirectional image.
Further, when the omnidirectional image is encoded in accordance with a general encoding scheme such as Moving Picture Experts Group phase 2 (MPEG 2) or Advanced Video Coding (AVC)/H.264, encoding is performed using information of the entire screen. Therefore, it is difficult to decode only the encoded stream of the area corresponding to the line-of-sight vector of the user from the encoded stream in which the omnidirectional image is coded as one image.
In this regard, a technique of reducing a decoding process amount by dividing and encoding an omnidirectional image through a recording device and decoding only the encoded streams of the divided areas corresponding to the line-of-sight vector of the user through a reproducing device is considered (for example, see Patent Literatures 2 and 3).