Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror
  Reflections
We tackle the problem of generating highly realistic and plausible mirror
reflections using diffusion-based generative models. We formulate this problem
as an image inpainting task, allowing for more user control over the placement
of mirrors during the generation process. To enable this, we create SynMirror,
a large-scale dataset of diverse synthetic scenes with objects placed in front
of mirrors. SynMirror contains around 198K samples rendered from 66K unique 3D
objects, along with their associated depth maps, normal maps and instance-wise
segmentation masks, to capture relevant geometric properties of the scene.
Using this dataset, we propose a novel depth-conditioned inpainting method
called MirrorFusion, which generates high-quality geometrically consistent and
photo-realistic mirror reflections given an input image and a mask depicting
the mirror region. MirrorFusion outperforms state-of-the-art methods on
SynMirror, as demonstrated by extensive quantitative and qualitative analysis.
To the best of our knowledge, we are the first to successfully tackle the
challenging problem of generating controlled and faithful mirror reflections of
an object in a scene using diffusion based models. SynMirror and MirrorFusion
open up new avenues for image editing and augmented reality applications for
practitioners and researchers alike.
66f2cd3663244bb5e670e52f
Ankit Dhiman
66f2cd3663244bb5e670e530
Manan Shah
claimed_verified
2024-09-25T07:33:44.279Z
633d366a748f94ec2da80e8b
/avatars/40f7f4d36727288a69ef7ea94fd3c853.svg
Manan Shah
user
cs-mshah
66f2cd3663244bb5e670e531
Rishubh Parihar
claimed_verified
2025-04-09T14:37:30.710Z
631af10b492c0c57a22ea3e2
https://cdn-avatars.huggingface.co/v1/production/uploads/1662710000523-noauth.png
Rishubh Parihar
user
RishubhPar
66f2cd3663244bb5e670e532
Yash Bhalgat
claimed_verified
2024-09-25T07:33:41.909Z
630918cc3dbb0cf0686bae1d
/avatars/607cc5ee6a70a283e8d6e2845ce15e41.svg
Yash Bhalgat
user
yashbhalgat
66f2cd3663244bb5e670e533
Lokesh R Boregowda
66f2cd3663244bb5e670e534
R Venkatesh Babu
Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror
  Reflections
We tackle the problem of generating highly realistic and plausible mirror
reflections using diffusion-based generative models. We formulate this problem
as an image inpainting task, allowing for more user control over the placement
of mirrors during the generation process. To enable this, we create SynMirror,
a large-scale dataset of diverse synthetic scenes with objects placed in front
of mirrors. SynMirror contains around 198K samples rendered from 66K unique 3D
objects, along with their associated depth maps, normal maps and instance-wise
segmentation masks, to capture relevant geometric properties of the scene.
Using this dataset, we propose a novel depth-conditioned inpainting method
called MirrorFusion, which generates high-quality geometrically consistent and
photo-realistic mirror reflections given an input image and a mask depicting
the mirror region. MirrorFusion outperforms state-of-the-art methods on
SynMirror, as demonstrated by extensive quantitative and qualitative analysis.
To the best of our knowledge, we are the first to successfully tackle the
challenging problem of generating controlled and faithful mirror reflections of
an object in a scene using diffusion based models. SynMirror and MirrorFusion
open up new avenues for image editing and augmented reality applications for
practitioners and researchers alike.
A diffusion-based generative model issues high-fidelity mirror reflections using a novel depth-conditioned inpainting method on a large synthetic dataset, enabling user control over mirror placement.
diffusion-based generative models image inpainting SynMirror depth-conditioned inpainting MirrorFusion geometrically consistent photo-realistic mirror reflections