PNVC: Towards Practical INR-based Video Compression
<|reference_start|>PNVC: Towards Practical INR-based Video Compression: Neural video compression has recently demonstrated significant potential to compete with conventional video codecs in terms of rate-quality performance. These learned video codecs are however associated with various issues related to decoding complexity (for autoencoder-based methods) and/or system delays (for implicit neural representation (INR) based models), which currently prevent them from being deployed in practical applications. In this paper, targeting a practical neural video codec, we propose a novel INR-based coding framework, PNVC, which innovatively combines autoencoder-based and overfitted solutions. Our approach benefits from several design innovations, including a new structural reparameterization-based architecture, hierarchical quality control, modulation-based entropy modeling, and scale-aware positional embedding. Supporting both low delay (LD) and random access (RA) configurations, PNVC outperforms existing INR-based codecs, achieving nearly 35%+ BD-rate savings against HEVC HM 18.0 (LD) - almost 10% more compared to one of the state-of-the-art INR-based codecs, HiNeRV and 5% more over VTM 20.0 (LD), while maintaining 20+ FPS decoding speeds for 1080p content. This represents an important step forward for INR-based video coding, moving it towards practical deployment. The source code will be available for public evaluation.<|reference_end|>
@article{gao2024pnvc:,
    title={PNVC: Towards Practical INR-based Video Compression},
    author={Ge Gao, Ho Man Kwan, Fan Zhang, David Bull},
    journal={arXiv preprint arXiv:2409.00953},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.00953},
    primaryClass={cs.CV}
}