MVDiffusion: Enabling Holistic Multi-view Image Generation with
  Correspondence-Aware Diffusion
This paper introduces MVDiffusion, a simple yet effective multi-view image
generation method for scenarios where pixel-to-pixel correspondences are
available, such as perspective crops from panorama or multi-view images given
geometry (depth maps and poses). Unlike prior models that rely on iterative
image warping and inpainting, MVDiffusion concurrently generates all images
with a global awareness, encompassing high resolution and rich content,
effectively addressing the error accumulation prevalent in preceding models.
MVDiffusion specifically incorporates a correspondence-aware attention
mechanism, enabling effective cross-view interaction. This mechanism underpins
three pivotal modules: 1) a generation module that produces low-resolution
images while maintaining global correspondence, 2) an interpolation module that
densifies spatial coverage between images, and 3) a super-resolution module
that upscales into high-resolution outputs. In terms of panoramic imagery,
MVDiffusion can generate high-resolution photorealistic images up to
1024times1024 pixels. For geometry-conditioned multi-view image generation,
MVDiffusion demonstrates the first method capable of generating a textured map
of a scene mesh. The project page is at https://mvdiffusion.github.io.
64a38e66aab63290742d194f
Shitao Tang
claimed_verified
2023-07-04T19:16:53.014Z
644f2b289b4e87c31bb19fbd
/avatars/b1c14324699b4f781097f53acacc864d.svg
Shitao Tang
user
tangshitao
64a38e66aab63290742d1950
Fuyang Zhang
64a38e66aab63290742d1951
Jiacheng Chen
claimed_verified
2024-06-15T07:46:57.838Z
655bca95360e4f90cb61ba83
/avatars/1a187beb91a5e2fdc2303620b742aab1.svg
Jiacheng Chen
user
cccjc
64a38e66aab63290742d1952
Peng Wang
claimed_verified
2023-07-04T19:16:48.780Z
619c4a5cb392787f0f3ead53
/avatars/e62bc6fed6e9cbe4e0339e050cc3be5f.svg
peng  wang
user
Peng-Wang
64a38e66aab63290742d1953
Yasutaka Furukawa
MVDiffusion: Enabling Holistic Multi-view Image Generation with
  Correspondence-Aware Diffusion
This paper introduces MVDiffusion, a simple yet effective multi-view image
generation method for scenarios where pixel-to-pixel correspondences are
available, such as perspective crops from panorama or multi-view images given
geometry (depth maps and poses). Unlike prior models that rely on iterative
image warping and inpainting, MVDiffusion concurrently generates all images
with a global awareness, encompassing high resolution and rich content,
effectively addressing the error accumulation prevalent in preceding models.
MVDiffusion specifically incorporates a correspondence-aware attention
mechanism, enabling effective cross-view interaction. This mechanism underpins
three pivotal modules: 1) a generation module that produces low-resolution
images while maintaining global correspondence, 2) an interpolation module that
densifies spatial coverage between images, and 3) a super-resolution module
that upscales into high-resolution outputs. In terms of panoramic imagery,
MVDiffusion can generate high-resolution photorealistic images up to
1024times1024 pixels. For geometry-conditioned multi-view image generation,
MVDiffusion demonstrates the first method capable of generating a textured map
of a scene mesh. The project page is at https://mvdiffusion.github.io.
MVDiffusion is a multi-view image generation method that uses a correspondence-aware attention mechanism to simultaneously create high-resolution images with global consistency and fine details.
multi-view image generation pixel-to-pixel correspondences iterative image warping inpainting global awareness correspondence-aware attention mechanism generation module interpolation module super-resolution module panoramic imagery photorealistic images