title: Video Instance Matting
summary: Conventional video matting outputs one alpha matte for all instances
  appearing in a video frame so that individual instances are not distinguished.
  While video instance segmentation provides time-consistent instance masks,
  results are unsatisfactory for matting applications, especially due to applied
  binarization. To remedy this deficiency, we propose Video Instance
  Matting~(VIM), that is, estimating alpha mattes of each instance at each frame
  of a video sequence. To tackle this challenging problem, we present MSG-VIM, a
  Mask Sequence Guided Video Instance Matting neural network, as a novel baseline
  model for VIM. MSG-VIM leverages a mixture of mask augmentations to make
  predictions robust to inaccurate and inconsistent mask guidance. It
  incorporates temporal mask and temporal feature guidance to improve the
  temporal consistency of alpha matte predictions. Furthermore, we build a new
  benchmark for VIM, called VIM50, which comprises 50 video clips with multiple
  human instances as foreground objects. To evaluate performances on the VIM
  task, we introduce a suitable metric called Video Instance-aware Matting
  Quality~(VIMQ). Our proposed model MSG-VIM sets a strong baseline on the VIM50
  benchmark and outperforms existing methods by a large margin. The project is
  open-sourced at https://github.com/SHI-Labs/VIM.
paper_authors._id: 654b115d2903299c2c636768
paper_authors.name: Jiachen Li
paper_authors._id: 654b115d2903299c2c636769
paper_authors.name: Roberto Henschel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:07:10.464Z
paper_authors._id: 6316145f95b55e2621d0a128
paper_authors.avatarUrl: /avatars/6a35dbd6f48b6e34a6fc8b8d04e00f6a.svg
paper_authors.fullname: Roberto Henschel
paper_authors.type: user
paper_authors.user: robhen
paper_authors._id: 654b115d2903299c2c63676a
paper_authors.name: Vidit Goel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:07:16.949Z
paper_authors._id: 636c0c1a15cd58e915bb8139
paper_authors.avatarUrl: /avatars/7c675ac6a7d303d3425e498c4e939eb0.svg
paper_authors.fullname: Vidit Goel
paper_authors.type: user
paper_authors.user: vidit98
paper_authors._id: 654b115d2903299c2c63676b
paper_authors.name: Marianna Ohanyan
paper_authors._id: 654b115d2903299c2c63676c
paper_authors.name: Shant Navasardyan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:07:28.913Z
paper_authors._id: 641dadf816ef0e5a7275a5cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641dadf816ef0e5a7275a5cd/YrieHABBK9VH82VxYPJnp.jpeg
paper_authors.fullname: Shant Navasardyan
paper_authors.type: user
paper_authors.user: navasardyanshant
paper_authors._id: 654b115d2903299c2c63676d
paper_authors.name: Humphrey Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T09:07:36.420Z
paper_authors._id: 61e1188afc27c0f5e3641eb3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61e1188afc27c0f5e3641eb3/R4JFlWgYomZFvZ9CLSZfD.jpeg
paper_authors.fullname: Humphrey Shi
paper_authors.type: user
paper_authors.user: Humphrey
paper_title: Video Instance Matting
paper_summary: Conventional video matting outputs one alpha matte for all instances
  appearing in a video frame so that individual instances are not distinguished.
  While video instance segmentation provides time-consistent instance masks,
  results are unsatisfactory for matting applications, especially due to applied
  binarization. To remedy this deficiency, we propose Video Instance
  Matting~(VIM), that is, estimating alpha mattes of each instance at each frame
  of a video sequence. To tackle this challenging problem, we present MSG-VIM, a
  Mask Sequence Guided Video Instance Matting neural network, as a novel baseline
  model for VIM. MSG-VIM leverages a mixture of mask augmentations to make
  predictions robust to inaccurate and inconsistent mask guidance. It
  incorporates temporal mask and temporal feature guidance to improve the
  temporal consistency of alpha matte predictions. Furthermore, we build a new
  benchmark for VIM, called VIM50, which comprises 50 video clips with multiple
  human instances as foreground objects. To evaluate performances on the VIM
  task, we introduce a suitable metric called Video Instance-aware Matting
  Quality~(VIMQ). Our proposed model MSG-VIM sets a strong baseline on the VIM50
  benchmark and outperforms existing methods by a large margin. The project is
  open-sourced at https://github.com/SHI-Labs/VIM.
paper_ai_summary: MSG-VIM, a neural network for video instance matting, improves temporal consistency in alpha matte predictions using mask and feature guidance, outperforming existing methods on the VIM50 benchmark.
paper_ai_keywords: mask sequence guided alpha mattes video instance matting mask augmentations temporal mask guidance temporal feature guidance Video Instance-aware Matting Quality VIMQ VIM50