Hybrid Cost Volume for Memory-Efficient Optical Flow
<|reference_start|>Hybrid Cost Volume for Memory-Efficient Optical Flow: Current state-of-the-art flow methods are mostly based on dense all-pairs cost volumes. However, as image resolution increases, the computational and spatial complexity of constructing these cost volumes grows at a quartic rate, making these methods impractical for high-resolution images. In this paper, we propose a novel Hybrid Cost Volume for memory-efficient optical flow, named HCV. To construct HCV, we first propose a Top-k strategy to separate the 4D cost volume into two global 3D cost volumes. These volumes significantly reduce memory usage while retaining a substantial amount of matching information. We further introduce a local 4D cost volume with a local search space to supplement the local information for HCV. Based on HCV, we design a memory-efficient optical flow network, named HCVFlow. Compared to the recurrent flow methods based the all-pairs cost volumes, our HCVFlow significantly reduces memory consumption while ensuring high accuracy. We validate the effectiveness and efficiency of our method on the Sintel and KITTI datasets and real-world 4K (2160*3840) resolution images. Extensive experiments show that our HCVFlow has very low memory usage and outperforms other memory-efficient methods in terms of accuracy. The code is publicly available at https://github.com/gangweiX/HCVFlow.<|reference_end|>
@article{zhao2024hybrid,
    title={Hybrid Cost Volume for Memory-Efficient Optical Flow},
    author={Yang Zhao, Gangwei Xu, Gang Wu},
    journal={arXiv preprint arXiv:2409.04243},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.04243},
    primaryClass={cs.CV}
}