Title: Hey all,<p>You&#x27;ve probably seen projects that add objects to an image from a style or text prompt, like InteriorAI (levelsio) and Adobe Firefly. The prevalent issue with these diffusion-based inpainting approaches is that they don&#x27;t yet have great conditioning on lighting, perspective, and structure. You&#x27;ll often get incorrect or generic shadows; warped-looking objects; and distorted backgrounds.<p>What is Fill 3D?
Fill 3D is an exploration on doing generative fill in 3D to render ultra-realistic results that harmonize with the background image, using industry-standard path tracing, akin to compositing in Hollywood movies.<p>How does it work?
1. Deproject: First, deproject an image to a 3D shell using both geometric and photometric cues from the input image.
2. Place: Draw rectangles and describe what you want in them, akin to Photoshop&#x27;s Generative Fill feature.
3. Render: Use good ol&#x27; path tracing to render ultra-realistic results.<p>Why Fill 3D?
+ The results are insanely realistic (see video in the github repo, or on the website).
+ Fast enough: Currently, generations take 40-80 seconds. Diffusion takes ~10seconds, so we&#x27;re slower, but for the level of realism, it&#x27;s pretty good.
+ Potential applications: I&#x27;m thinking of virtual staging in real estate media, what do you think?<p>Check it out at <a href="https:&#x2F;&#x2F;fill3d.ai" rel="nofollow noreferrer">https:&#x2F;&#x2F;fill3d.ai</a>
+ There&#x27;s API access! :D
+ Right now, you need an image of an empty room. Will loosen this restriction over time.<p>Fill 3D is built on Function (<a href="https:&#x2F;&#x2F;fxn.ai" rel="nofollow noreferrer">https:&#x2F;&#x2F;fxn.ai</a>). With Function, I can run the Python functions that do the steps above on powerful GPUs with only code (no Dockerfile, YAML, k8s, etc), and invoke them from just about anywhere. I&#x27;m the founder of fxn.<p>Tell me what you think!!<p>PS: This is my first Show HN, so please be nice :)
Upvote: