title: OmnimatteRF: Robust Omnimatte with 3D Background Modeling
summary: Video matting has broad applications, from adding interesting effects to
  casually captured movies to assisting video production professionals. Matting
  with associated effects such as shadows and reflections has also attracted
  increasing research activity, and methods like Omnimatte have been proposed to
  separate dynamic foreground objects of interest into their own layers. However,
  prior works represent video backgrounds as 2D image layers, limiting their
  capacity to express more complicated scenes, thus hindering application to
  real-world videos. In this paper, we propose a novel video matting method,
  OmnimatteRF, that combines dynamic 2D foreground layers and a 3D background
  model. The 2D layers preserve the details of the subjects, while the 3D
  background robustly reconstructs scenes in real-world videos. Extensive
  experiments demonstrate that our method reconstructs scenes with better quality
  on various videos.
paper_authors._id: 6503fa9724964747c8da58c1
paper_authors.name: Geng Lin
paper_authors._id: 6503fa9724964747c8da58c2
paper_authors.name: Chen Gao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-17T19:21:03.708Z
paper_authors._id: 65049f41c6ae3df8f28cfe96
paper_authors.avatarUrl: /avatars/a653ed4a215c4bd34710a1cee9f4d9cd.svg
paper_authors.fullname: Chen Gao
paper_authors.type: user
paper_authors.user: gaochen315
paper_authors._id: 6503fa9724964747c8da58c3
paper_authors.name: Jia-Bin Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T08:57:51.227Z
paper_authors._id: 641c139b73296f7ee256970c
paper_authors.avatarUrl: /avatars/5a2550d95e686640242840ad3bd0e680.svg
paper_authors.fullname: Jiabin Huang
paper_authors.type: user
paper_authors.user: YellowAddice
paper_authors._id: 6503fa9724964747c8da58c4
paper_authors.name: Changil Kim
paper_authors._id: 6503fa9724964747c8da58c5
paper_authors.name: Yipeng Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T09:00:15.581Z
paper_authors._id: 63bed1ca24e75d1efdfac4f6
paper_authors.avatarUrl: /avatars/8243d88ce697bb6a0f1a585402fd3029.svg
paper_authors.fullname: Yipeng Wang
paper_authors.type: user
paper_authors.user: promyse
paper_authors._id: 6503fa9724964747c8da58c6
paper_authors.name: Matthias Zwicker
paper_authors._id: 6503fa9724964747c8da58c7
paper_authors.name: Ayush Saraf
paper_title: OmnimatteRF: Robust Omnimatte with 3D Background Modeling
paper_summary: Video matting has broad applications, from adding interesting effects to
  casually captured movies to assisting video production professionals. Matting
  with associated effects such as shadows and reflections has also attracted
  increasing research activity, and methods like Omnimatte have been proposed to
  separate dynamic foreground objects of interest into their own layers. However,
  prior works represent video backgrounds as 2D image layers, limiting their
  capacity to express more complicated scenes, thus hindering application to
  real-world videos. In this paper, we propose a novel video matting method,
  OmnimatteRF, that combines dynamic 2D foreground layers and a 3D background
  model. The 2D layers preserve the details of the subjects, while the 3D
  background robustly reconstructs scenes in real-world videos. Extensive
  experiments demonstrate that our method reconstructs scenes with better quality
  on various videos.
paper_ai_keywords: dynamic 2D foreground layers 3D background model video matting OmnimatteRF shadows reflections