title: Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
summary: This paper presents Paint3D, a novel coarse-to-fine generative framework that
  is capable of producing high-resolution, lighting-less, and diverse 2K UV
  texture maps for untextured 3D meshes conditioned on text or image inputs. The
  key challenge addressed is generating high-quality textures without embedded
  illumination information, which allows the textures to be re-lighted or
  re-edited within modern graphics pipelines. To achieve this, our method first
  leverages a pre-trained depth-aware 2D diffusion model to generate
  view-conditional images and perform multi-view texture fusion, producing an
  initial coarse texture map. However, as 2D models cannot fully represent 3D
  shapes and disable lighting effects, the coarse texture map exhibits incomplete
  areas and illumination artifacts. To resolve this, we train separate UV
  Inpainting and UVHD diffusion models specialized for the shape-aware refinement
  of incomplete areas and the removal of illumination artifacts. Through this
  coarse-to-fine process, Paint3D can produce high-quality 2K UV textures that
  maintain semantic consistency while being lighting-less, significantly
  advancing the state-of-the-art in texturing 3D objects.
paper_authors._id: 6584f4d235c3b05065cee8b0
paper_authors.name: Xianfang Zeng
paper_authors._id: 6585cc669747b6c745a14ef2
paper_authors.name: Xin Chen
paper_authors._id: 6585cc899747b6c745a14ef3
paper_authors.name: Zhongqi Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-05T07:25:19.046Z
paper_authors._id: 63e3b7bec65f975b436d12be
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e3b7bec65f975b436d12be/guGZu6TBQJMBcrbaigjag.jpeg
paper_authors.fullname: Qi
paper_authors.type: user
paper_authors.user: GeorgeQi
paper_authors._id: 6585cca09747b6c745a14ef4
paper_authors.name: Wen Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-22T10:31:47.451Z
paper_authors._id: 63198e8802fb322037332f2d
paper_authors.avatarUrl: /avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
paper_authors.fullname: Wen Liu
paper_authors.type: user
paper_authors.user: doubility123
paper_authors._id: 6585ccb79747b6c745a14ef5
paper_authors.name: Zibo Zhao
paper_authors._id: 6585ccd39747b6c745a14ef6
paper_authors.name: Zhibin Wang
paper_authors._id: 6585ccfc9747b6c745a14ef8
paper_authors.name: BIN FU
paper_authors._id: 6585cd3a9747b6c745a14ef9
paper_authors.name: Yong Liu
paper_authors._id: 6585cd539747b6c745a14efa
paper_authors.name: Gang Yu
paper_title: Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
paper_summary: This paper presents Paint3D, a novel coarse-to-fine generative framework that
  is capable of producing high-resolution, lighting-less, and diverse 2K UV
  texture maps for untextured 3D meshes conditioned on text or image inputs. The
  key challenge addressed is generating high-quality textures without embedded
  illumination information, which allows the textures to be re-lighted or
  re-edited within modern graphics pipelines. To achieve this, our method first
  leverages a pre-trained depth-aware 2D diffusion model to generate
  view-conditional images and perform multi-view texture fusion, producing an
  initial coarse texture map. However, as 2D models cannot fully represent 3D
  shapes and disable lighting effects, the coarse texture map exhibits incomplete
  areas and illumination artifacts. To resolve this, we train separate UV
  Inpainting and UVHD diffusion models specialized for the shape-aware refinement
  of incomplete areas and the removal of illumination artifacts. Through this
  coarse-to-fine process, Paint3D can produce high-quality 2K UV textures that
  maintain semantic consistency while being lighting-less, significantly
  advancing the state-of-the-art in texturing 3D objects.
paper_ai_keywords: depth-aware 2D diffusion model multi-view texture fusion UV Inpainting UVHD diffusion models shape-aware refinement semantic consistency