Display of a volume of captured video or positional tracking video may enable a viewer to perceive a captured scene from any location and at any viewing angle within a viewing volume. Using the data provided by such a video system, a viewpoint can be reconstructed to provide full motion parallax and/or correct view-dependent lighting. When viewing this video with a virtual reality head-mounted display, the user may enjoy an immersive virtual presence within an environment. Such a virtual reality experience may provide six degrees of freedom and uncompromised stereoscopic perception at any interpupillary distance.
One key challenge to video with a three-dimensional viewing volume is its immense data volume, which may be more than 100 times larger than that of conventional two-dimensional video. The amount of data needed may grow exponentially as the viewing volume increases. The large data requirement may become prohibitive for viewers to store the content in consumer-grade devices or for distributors to transmit the content over a network. Meeting memory, bandwidth, and/or processing requirements may also be challenges for low-latency and high-fidelity playback in today's computer. To address these challenges, an efficient compression and playback scheme is needed.