title: ZeroComp: Zero-shot Object Compositing from Image Intrinsics via
    Diffusion
summary: We present ZeroComp, an effective zero-shot 3D object compositing approach
  that does not require paired composite-scene images during training. Our method
  leverages ControlNet to condition from intrinsic images and combines it with a
  Stable Diffusion model to utilize its scene priors, together operating as an
  effective rendering engine. During training, ZeroComp uses intrinsic images
  based on geometry, albedo, and masked shading, all without the need for paired
  images of scenes with and without composite objects. Once trained, it
  seamlessly integrates virtual 3D objects into scenes, adjusting shading to
  create realistic composites. We developed a high-quality evaluation dataset and
  demonstrate that ZeroComp outperforms methods using explicit lighting
  estimations and generative techniques in quantitative and human perception
  benchmarks. Additionally, ZeroComp extends to real and outdoor image
  compositing, even when trained solely on synthetic indoor data, showcasing its
  effectiveness in image compositing.
paper_authors._id: 670aa68d4bea0a5a627c6c23
paper_authors.name: Zitian Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T13:17:10.791Z
paper_authors._id: 655dbda6668b64adf1495528
paper_authors.avatarUrl: /avatars/dbf6072dcf0905873289d78209a92f87.svg
paper_authors.fullname: Zitian Zhang
paper_authors.type: user
paper_authors.user: zzt76
paper_authors._id: 670aa68d4bea0a5a627c6c24
paper_authors.name: Frédéric Fortier-Chouinard
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:15:26.881Z
paper_authors._id: 64763956fb22e3b77f3e13d6
paper_authors.avatarUrl: /avatars/1e6acf5d733280931ba2ce099481b2be.svg
paper_authors.fullname: Frédéric Fortier-Chouinard
paper_authors.type: user
paper_authors.user: lefreud
paper_authors._id: 670aa68d4bea0a5a627c6c25
paper_authors.name: Mathieu Garon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T13:17:05.387Z
paper_authors._id: 657df7d22bffc5568ae27a91
paper_authors.avatarUrl: /avatars/a56fc5d1faaaff4b2b44f0d2c6e54ced.svg
paper_authors.fullname: Mathieu Garon
paper_authors.type: user
paper_authors.user: MathieuGaron
paper_authors._id: 670aa68d4bea0a5a627c6c26
paper_authors.name: Anand Bhattad
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T13:16:30.640Z
paper_authors._id: 6303d5924ec2dfa82a5627de
paper_authors.avatarUrl: /avatars/1e433b7fb9d30b9f42dec487d658b1e3.svg
paper_authors.fullname: Anand Bhattad
paper_authors.type: user
paper_authors.user: anandbhattad
paper_authors._id: 670aa68d4bea0a5a627c6c27
paper_authors.name: Jean-François Lalonde
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T13:16:36.336Z
paper_authors._id: 66e9905f00d5cd69e36e0e1a
paper_authors.avatarUrl: /avatars/082eb98a634ae2f46c3664c2e6d331a1.svg
paper_authors.fullname: Jean-Francois Lalonde
paper_authors.type: user
paper_authors.user: jflalonde
paper_title: ZeroComp: Zero-shot Object Compositing from Image Intrinsics via
    Diffusion
paper_summary: We present ZeroComp, an effective zero-shot 3D object compositing approach
  that does not require paired composite-scene images during training. Our method
  leverages ControlNet to condition from intrinsic images and combines it with a
  Stable Diffusion model to utilize its scene priors, together operating as an
  effective rendering engine. During training, ZeroComp uses intrinsic images
  based on geometry, albedo, and masked shading, all without the need for paired
  images of scenes with and without composite objects. Once trained, it
  seamlessly integrates virtual 3D objects into scenes, adjusting shading to
  create realistic composites. We developed a high-quality evaluation dataset and
  demonstrate that ZeroComp outperforms methods using explicit lighting
  estimations and generative techniques in quantitative and human perception
  benchmarks. Additionally, ZeroComp extends to real and outdoor image
  compositing, even when trained solely on synthetic indoor data, showcasing its
  effectiveness in image compositing.
paper_ai_summary: ZeroComp combines ControlNet and Stable Diffusion to achieve zero-shot 3D object compositing without requiring paired training images, demonstrating superior performance in various benchmarks.
paper_ai_keywords: ControlNet Stable Diffusion intrinsic images geometry albedo masked shading rendering engine evaluation dataset explicit lighting estimations generative techniques indoor data real-world images outdoor image compositing