title: MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
summary: Texturing is a crucial step in the 3D asset production workflow, which
  enhances the visual appeal and diversity of 3D assets. Despite recent
  advancements in Text-to-Texture (T2T) generation, existing methods often yield
  subpar results, primarily due to local discontinuities, inconsistencies across
  multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle
  these challenges, we propose a novel generation-refinement 3D texturing
  framework called MVPaint, which can generate high-resolution, seamless textures
  while emphasizing multi-view consistency. MVPaint mainly consists of three key
  modules. 1) Synchronized Multi-view Generation (SMG). Given a 3D mesh model,
  MVPaint first simultaneously generates multi-view images by employing an SMG
  model, which leads to coarse texturing results with unpainted parts due to
  missing observations. 2) Spatial-aware 3D Inpainting (S3I). To ensure complete
  3D texturing, we introduce the S3I method, specifically designed to effectively
  texture previously unobserved areas. 3) UV Refinement (UVR). Furthermore,
  MVPaint employs a UVR module to improve the texture quality in the UV space,
  which first performs a UV-space Super-Resolution, followed by a Spatial-aware
  Seam-Smoothing algorithm for revising spatial texturing discontinuities caused
  by UV unwrapping. Moreover, we establish two T2T evaluation benchmarks: the
  Objaverse T2T benchmark and the GSO T2T benchmark, based on selected
  high-quality 3D meshes from the Objaverse dataset and the entire GSO dataset,
  respectively. Extensive experimental results demonstrate that MVPaint surpasses
  existing state-of-the-art methods. Notably, MVPaint could generate
  high-fidelity textures with minimal Janus issues and highly enhanced cross-view
  consistency.
paper_authors._id: 672a3b03d0448c8bae6dc97b
paper_authors.name: Wei Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T16:09:12.427Z
paper_authors._id: 64b914c8ace99c0723ad83a9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b914c8ace99c0723ad83a9/udUHjj6fby82zh8LDjXhL.jpeg
paper_authors.fullname: Wei Cheng
paper_authors.type: user
paper_authors.user: wchengad
paper_authors._id: 672a3b03d0448c8bae6dc97c
paper_authors.name: Juncheng Mu
paper_authors._id: 672a3b03d0448c8bae6dc97d
paper_authors.name: Xianfang Zeng
paper_authors._id: 672a3b03d0448c8bae6dc97e
paper_authors.name: Xin Chen
paper_authors._id: 672a3b03d0448c8bae6dc97f
paper_authors.name: Anqi Pang
paper_authors._id: 672a3b03d0448c8bae6dc980
paper_authors.name: Chi Zhang
paper_authors._id: 672a3b03d0448c8bae6dc981
paper_authors.name: Zhibin Wang
paper_authors._id: 672a3b03d0448c8bae6dc982
paper_authors.name: Bin Fu
paper_authors._id: 672a3b03d0448c8bae6dc983
paper_authors.name: Gang Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:37:39.626Z
paper_authors._id: 63417332c5565a4b8d43a0d8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63417332c5565a4b8d43a0d8/MmnYG7Wu2Z_lqCBvTfJmy.png
paper_authors.fullname: Gang Yu
paper_authors.type: user
paper_authors.user: skicy
paper_authors._id: 672a3b03d0448c8bae6dc984
paper_authors.name: Ziwei Liu
paper_authors._id: 672a3b03d0448c8bae6dc985
paper_authors.name: Liang Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T16:09:10.438Z
paper_authors._id: 64ef30fb0de94d31b7921175
paper_authors.avatarUrl: /avatars/d4f4c7be460befc1cedec13bbf9db972.svg
paper_authors.fullname: pan
paper_authors.type: user
paper_authors.user: pldeqiushui
paper_title: MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
paper_summary: Texturing is a crucial step in the 3D asset production workflow, which
  enhances the visual appeal and diversity of 3D assets. Despite recent
  advancements in Text-to-Texture (T2T) generation, existing methods often yield
  subpar results, primarily due to local discontinuities, inconsistencies across
  multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle
  these challenges, we propose a novel generation-refinement 3D texturing
  framework called MVPaint, which can generate high-resolution, seamless textures
  while emphasizing multi-view consistency. MVPaint mainly consists of three key
  modules. 1) Synchronized Multi-view Generation (SMG). Given a 3D mesh model,
  MVPaint first simultaneously generates multi-view images by employing an SMG
  model, which leads to coarse texturing results with unpainted parts due to
  missing observations. 2) Spatial-aware 3D Inpainting (S3I). To ensure complete
  3D texturing, we introduce the S3I method, specifically designed to effectively
  texture previously unobserved areas. 3) UV Refinement (UVR). Furthermore,
  MVPaint employs a UVR module to improve the texture quality in the UV space,
  which first performs a UV-space Super-Resolution, followed by a Spatial-aware
  Seam-Smoothing algorithm for revising spatial texturing discontinuities caused
  by UV unwrapping. Moreover, we establish two T2T evaluation benchmarks: the
  Objaverse T2T benchmark and the GSO T2T benchmark, based on selected
  high-quality 3D meshes from the Objaverse dataset and the entire GSO dataset,
  respectively. Extensive experimental results demonstrate that MVPaint surpasses
  existing state-of-the-art methods. Notably, MVPaint could generate
  high-fidelity textures with minimal Janus issues and highly enhanced cross-view
  consistency.
paper_ai_keywords: Synchronized Multi-view Generation (SMG) Spatial-aware 3D Inpainting (S3I) UV Refinement (UVR) Super-Resolution Spatial-aware Seam-Smoothing T2T evaluation benchmarks Objaverse T2T benchmark GSO T2T benchmark Janus issues