title: RealFill: Reference-Driven Generation for Authentic Image Completion
summary: Recent advances in generative imagery have brought forth outpainting and
  inpainting models that can produce high-quality, plausible image content in
  unknown regions, but the content these models hallucinate is necessarily
  inauthentic, since the models lack sufficient context about the true scene. In
  this work, we propose RealFill, a novel generative approach for image
  completion that fills in missing regions of an image with the content that
  should have been there. RealFill is a generative inpainting model that is
  personalized using only a few reference images of a scene. These reference
  images do not have to be aligned with the target image, and can be taken with
  drastically varying viewpoints, lighting conditions, camera apertures, or image
  styles. Once personalized, RealFill is able to complete a target image with
  visually compelling contents that are faithful to the original scene. We
  evaluate RealFill on a new image completion benchmark that covers a set of
  diverse and challenging scenarios, and find that it outperforms existing
  approaches by a large margin. See more results on our project page:
  https://realfill.github.io
paper_authors._id: 65163399a546159af98e463f
paper_authors.name: Luming Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-02T06:59:52.224Z
paper_authors._id: 63e495a99db5da2dc1f679df
paper_authors.avatarUrl: /avatars/e3b3f7702a5c8fd21a246272940e45b8.svg
paper_authors.fullname: Luming Tang
paper_authors.type: user
paper_authors.user: lt453
paper_authors._id: 65163399a546159af98e4640
paper_authors.name: Nataniel Ruiz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-05T06:34:27.020Z
paper_authors._id: 6318c6e6cb116eab31d70a16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg
paper_authors.fullname: Nataniel
paper_authors.type: user
paper_authors.user: flavoredquark
paper_authors._id: 65163399a546159af98e4641
paper_authors.name: Qinghao Chu
paper_authors._id: 65163399a546159af98e4642
paper_authors.name: Yuanzhen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-29T12:06:54.260Z
paper_authors._id: 6334ee5831a2be3938c85639
paper_authors.avatarUrl: /avatars/9821af26a585f54b595ec39f061f2250.svg
paper_authors.fullname: Yuanzhen Li
paper_authors.type: user
paper_authors.user: yzli
paper_authors._id: 65163399a546159af98e4643
paper_authors.name: Aleksander Holynski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-29T12:06:39.178Z
paper_authors._id: 62f6e2792e53c2efd33faa92
paper_authors.avatarUrl: /avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
paper_authors.fullname: Aleksander Holynski
paper_authors.type: user
paper_authors.user: holynski
paper_authors._id: 65163399a546159af98e4644
paper_authors.name: David E. Jacobs
paper_authors._id: 65163399a546159af98e4645
paper_authors.name: Bharath Hariharan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-29T12:05:51.897Z
paper_authors._id: 64de7f10d7605441a8b348b2
paper_authors.avatarUrl: /avatars/177b5b386b416344dca325af185fb2c6.svg
paper_authors.fullname: Bharath Hariharan
paper_authors.type: user
paper_authors.user: bharathh
paper_authors._id: 65163399a546159af98e4646
paper_authors.name: Yael Pritch
paper_authors._id: 65163399a546159af98e4647
paper_authors.name: Neal Wadhwa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-29T12:05:33.067Z
paper_authors._id: 63041bb6eedc089484c82ed3
paper_authors.avatarUrl: /avatars/fca4b823f748cde5f5cff1212213523d.svg
paper_authors.fullname: Neal Wadhwa
paper_authors.type: user
paper_authors.user: NealWadhwa
paper_authors._id: 65163399a546159af98e4648
paper_authors.name: Kfir Aberman
paper_authors._id: 65163399a546159af98e4649
paper_authors.name: Michael Rubinstein
paper_title: RealFill: Reference-Driven Generation for Authentic Image Completion
paper_summary: Recent advances in generative imagery have brought forth outpainting and
  inpainting models that can produce high-quality, plausible image content in
  unknown regions, but the content these models hallucinate is necessarily
  inauthentic, since the models lack sufficient context about the true scene. In
  this work, we propose RealFill, a novel generative approach for image
  completion that fills in missing regions of an image with the content that
  should have been there. RealFill is a generative inpainting model that is
  personalized using only a few reference images of a scene. These reference
  images do not have to be aligned with the target image, and can be taken with
  drastically varying viewpoints, lighting conditions, camera apertures, or image
  styles. Once personalized, RealFill is able to complete a target image with
  visually compelling contents that are faithful to the original scene. We
  evaluate RealFill on a new image completion benchmark that covers a set of
  diverse and challenging scenarios, and find that it outperforms existing
  approaches by a large margin. See more results on our project page:
  https://realfill.github.io
paper_ai_keywords: generative imagery outpainting inpainting convolution synthesis image completion batch and sequential Bayes updates generative inpainting model reference images viewpoint lighting conditions camera apertures image styles visually compelling contents faithful to the original scene image completion benchmark