Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
This paper presents Paint3D, a novel coarse-to-fine generative framework that
is capable of producing high-resolution, lighting-less, and diverse 2K UV
texture maps for untextured 3D meshes conditioned on text or image inputs. The
key challenge addressed is generating high-quality textures without embedded
illumination information, which allows the textures to be re-lighted or
re-edited within modern graphics pipelines. To achieve this, our method first
leverages a pre-trained depth-aware 2D diffusion model to generate
view-conditional images and perform multi-view texture fusion, producing an
initial coarse texture map. However, as 2D models cannot fully represent 3D
shapes and disable lighting effects, the coarse texture map exhibits incomplete
areas and illumination artifacts. To resolve this, we train separate UV
Inpainting and UVHD diffusion models specialized for the shape-aware refinement
of incomplete areas and the removal of illumination artifacts. Through this
coarse-to-fine process, Paint3D can produce high-quality 2K UV textures that
maintain semantic consistency while being lighting-less, significantly
advancing the state-of-the-art in texturing 3D objects.
6584f4d235c3b05065cee8b0
Xianfang Zeng
6585cc669747b6c745a14ef2
Xin Chen
6585cc899747b6c745a14ef3
Zhongqi Qi
claimed_verified
2024-09-05T07:25:19.046Z
63e3b7bec65f975b436d12be
https://cdn-avatars.huggingface.co/v1/production/uploads/63e3b7bec65f975b436d12be/guGZu6TBQJMBcrbaigjag.jpeg
Qi
user
GeorgeQi
6585cca09747b6c745a14ef4
Wen Liu
claimed_verified
2024-01-22T10:31:47.451Z
63198e8802fb322037332f2d
/avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
Wen Liu
user
doubility123
6585ccb79747b6c745a14ef5
Zibo Zhao
6585ccd39747b6c745a14ef6
Zhibin Wang
6585ccfc9747b6c745a14ef8
BIN FU
6585cd3a9747b6c745a14ef9
Yong Liu
6585cd539747b6c745a14efa
Gang Yu
Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
This paper presents Paint3D, a novel coarse-to-fine generative framework that
is capable of producing high-resolution, lighting-less, and diverse 2K UV
texture maps for untextured 3D meshes conditioned on text or image inputs. The
key challenge addressed is generating high-quality textures without embedded
illumination information, which allows the textures to be re-lighted or
re-edited within modern graphics pipelines. To achieve this, our method first
leverages a pre-trained depth-aware 2D diffusion model to generate
view-conditional images and perform multi-view texture fusion, producing an
initial coarse texture map. However, as 2D models cannot fully represent 3D
shapes and disable lighting effects, the coarse texture map exhibits incomplete
areas and illumination artifacts. To resolve this, we train separate UV
Inpainting and UVHD diffusion models specialized for the shape-aware refinement
of incomplete areas and the removal of illumination artifacts. Through this
coarse-to-fine process, Paint3D can produce high-quality 2K UV textures that
maintain semantic consistency while being lighting-less, significantly
advancing the state-of-the-art in texturing 3D objects.
Paint3D uses a coarse-to-fine generative framework with specialized diffusion models to produce high-resolution, lighting-less UV texture maps for 3D meshes, addressing challenges of incomplete areas and illumination artifacts.
coarse-to-fine generative framework diffusion model UV texture maps 3D meshes 2D diffusion model view-conditional images multi-view texture fusion UV Inpainting UVHD diffusion models shape-aware refinement semantic consistency lighting-less