  To address the challenging portrait video matting problem more precisely,
existing works typically apply some matting priors that require additional user
efforts to obtain, such as annotated trimaps or background images. In this
work, we observe that instead of asking the user to explicitly provide a
background image, we may recover it from the input video itself. To this end,
we first propose a novel background restoration module (BRM) to recover the
background image dynamically from the input video. BRM is extremely lightweight
and can be easily integrated into existing matting models. By combining BRM
with a recent image matting model, MODNet, we then present MODNet-V for
portrait video matting. Benefited from the strong background prior provided by
BRM, MODNet-V has only 1/3 of the parameters of MODNet but achieves comparable
or even better performances. Our design allows MODNet-V to be trained in an
end-to-end manner on a single NVIDIA 3090 GPU. Finally, we introduce a new
patch refinement module (PRM) to adapt MODNet-V for high-resolution videos
while keeping MODNet-V lightweight and fast.

Computer Vision and Pattern Recognition