  Optical flow estimation with occlusion or large displacement is a problematic
challenge due to the lost of corresponding pixels between consecutive frames.
In this paper, we discover that the lost information is related to a large
quantity of motion features (more than 40%) computed from the popular
discriminative cost-volume feature would completely vanish due to invalid
sampling, leading to the low efficiency of optical flow learning. We call this
phenomenon the Vanishing Cost Volume Problem. Inspired by the fact that local
motion tends to be highly consistent within a short temporal window, we propose
a novel iterative Motion Feature Recovery (MFR) method to address the vanishing
cost volume via modeling motion consistency across multiple frames. In each MFR
iteration, invalid entries from original motion features are first determined
based on the current flow. Then, an efficient network is designed to adaptively
learn the motion correlation to recover invalid features for lost-information
restoration. The final optical flow is then decoded from the recovered motion
features. Experimental results on Sintel and KITTI show that our method
achieves state-of-the-art performances. In fact, MFR currently ranks second on
Sintel public website.

Computer Vision and Pattern Recognition