title: MVDiffusion: Enabling Holistic Multi-view Image Generation with
    Correspondence-Aware Diffusion
summary: This paper introduces MVDiffusion, a simple yet effective multi-view image
  generation method for scenarios where pixel-to-pixel correspondences are
  available, such as perspective crops from panorama or multi-view images given
  geometry (depth maps and poses). Unlike prior models that rely on iterative
  image warping and inpainting, MVDiffusion concurrently generates all images
  with a global awareness, encompassing high resolution and rich content,
  effectively addressing the error accumulation prevalent in preceding models.
  MVDiffusion specifically incorporates a correspondence-aware attention
  mechanism, enabling effective cross-view interaction. This mechanism underpins
  three pivotal modules: 1) a generation module that produces low-resolution
  images while maintaining global correspondence, 2) an interpolation module that
  densifies spatial coverage between images, and 3) a super-resolution module
  that upscales into high-resolution outputs. In terms of panoramic imagery,
  MVDiffusion can generate high-resolution photorealistic images up to
  1024times1024 pixels. For geometry-conditioned multi-view image generation,
  MVDiffusion demonstrates the first method capable of generating a textured map
  of a scene mesh. The project page is at https://mvdiffusion.github.io.
paper_authors._id: 64a38e66aab63290742d194f
paper_authors.name: Shitao Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T19:16:53.014Z
paper_authors._id: 644f2b289b4e87c31bb19fbd
paper_authors.avatarUrl: /avatars/b1c14324699b4f781097f53acacc864d.svg
paper_authors.fullname: Shitao Tang
paper_authors.type: user
paper_authors.user: tangshitao
paper_authors._id: 64a38e66aab63290742d1950
paper_authors.name: Fuyang Zhang
paper_authors._id: 64a38e66aab63290742d1951
paper_authors.name: Jiacheng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-15T07:46:57.838Z
paper_authors._id: 655bca95360e4f90cb61ba83
paper_authors.avatarUrl: /avatars/1a187beb91a5e2fdc2303620b742aab1.svg
paper_authors.fullname: Jiacheng Chen
paper_authors.type: user
paper_authors.user: cccjc
paper_authors._id: 64a38e66aab63290742d1952
paper_authors.name: Peng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T19:16:48.780Z
paper_authors._id: 619c4a5cb392787f0f3ead53
paper_authors.avatarUrl: /avatars/e62bc6fed6e9cbe4e0339e050cc3be5f.svg
paper_authors.fullname: peng  wang
paper_authors.type: user
paper_authors.user: Peng-Wang
paper_authors._id: 64a38e66aab63290742d1953
paper_authors.name: Yasutaka Furukawa
paper_title: MVDiffusion: Enabling Holistic Multi-view Image Generation with
    Correspondence-Aware Diffusion
paper_summary: This paper introduces MVDiffusion, a simple yet effective multi-view image
  generation method for scenarios where pixel-to-pixel correspondences are
  available, such as perspective crops from panorama or multi-view images given
  geometry (depth maps and poses). Unlike prior models that rely on iterative
  image warping and inpainting, MVDiffusion concurrently generates all images
  with a global awareness, encompassing high resolution and rich content,
  effectively addressing the error accumulation prevalent in preceding models.
  MVDiffusion specifically incorporates a correspondence-aware attention
  mechanism, enabling effective cross-view interaction. This mechanism underpins
  three pivotal modules: 1) a generation module that produces low-resolution
  images while maintaining global correspondence, 2) an interpolation module that
  densifies spatial coverage between images, and 3) a super-resolution module
  that upscales into high-resolution outputs. In terms of panoramic imagery,
  MVDiffusion can generate high-resolution photorealistic images up to
  1024times1024 pixels. For geometry-conditioned multi-view image generation,
  MVDiffusion demonstrates the first method capable of generating a textured map
  of a scene mesh. The project page is at https://mvdiffusion.github.io.
paper_ai_summary: MVDiffusion is a multi-view image generation method that uses a correspondence-aware attention mechanism to simultaneously create high-resolution images with global consistency and fine details.
paper_ai_keywords: multi-view image generation pixel-to-pixel correspondences iterative image warping inpainting global awareness correspondence-aware attention mechanism generation module interpolation module super-resolution module panoramic imagery photorealistic images