Bits-to-Photon: End-to-End Learned Scalable Point Cloud Compression for Direct Rendering
<|reference_start|>Bits-to-Photon: End-to-End Learned Scalable Point Cloud Compression for Direct Rendering: Point cloud is a promising 3D representation for volumetric streaming in emerging AR/VR applications. Despite recent advances in point cloud compression, decoding and rendering high-quality images from lossy compressed point clouds is still challenging in terms of quality and complexity, making it a major roadblock to achieve real-time 6-Degree-of-Freedom video streaming. In this paper, we address this problem by developing a point cloud compression scheme that generates a bit stream that can be directly decoded to renderable 3D Gaussians. The encoder and decoder are jointly optimized to consider both bit-rates and rendering quality. It significantly improves the rendering quality while substantially reducing decoding and rendering time, compared to existing point cloud compression methods. Furthermore, the proposed scheme generates a scalable bit stream, allowing multiple levels of details at different bit-rate ranges. Our method supports real-time color decoding and rendering of high quality point clouds, thus paving the way for interactive 3D streaming applications with free view points.<|reference_end|>
@article{hu2024bits-to-photon:,
    title={Bits-to-Photon: End-to-End Learned Scalable Point Cloud Compression for
  Direct Rendering},
    author={Yueyu Hu and Ran Gong and Yao Wang},
    journal={arXiv preprint arXiv:2406.05915},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05915},
    primaryClass={cs.CV eess.IV}
}