TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and
  Resampling
Given a 3D mesh, we aim to synthesize 3D textures that correspond to
arbitrary textual descriptions. Current methods for generating and assembling
textures from sampled views often result in prominent seams or excessive
smoothing. To tackle these issues, we present TexGen, a novel multi-view
sampling and resampling framework for texture generation leveraging a
pre-trained text-to-image diffusion model. For view consistent sampling, first
of all we maintain a texture map in RGB space that is parameterized by the
denoising step and updated after each sampling step of the diffusion model to
progressively reduce the view discrepancy. An attention-guided multi-view
sampling strategy is exploited to broadcast the appearance information across
views. To preserve texture details, we develop a noise resampling technique
that aids in the estimation of noise, generating inputs for subsequent
denoising steps, as directed by the text prompt and current texture map.
Through an extensive amount of qualitative and quantitative evaluations, we
demonstrate that our proposed method produces significantly better texture
quality for diverse 3D objects with a high degree of view consistency and rich
appearance details, outperforming current state-of-the-art methods.
Furthermore, our proposed texture generation technique can also be applied to
texture editing while preserving the original identity. More experimental
results are available at https://dong-huo.github.io/TexGen/
66b03b6f06c013ef866bca28
Dong Huo
claimed_verified
2024-08-05T14:33:34.474Z
64ca6be076200ec80fce102f
/avatars/e587d5bc22edf90aae4e6a681ffc56a8.svg
Philip Huo
user
Philip-Huo
66b03b6f06c013ef866bca29
Zixin Guo
66b03b6f06c013ef866bca2a
Xinxin Zuo
66b03b6f06c013ef866bca2b
Zhihao Shi
admin_assigned
2024-08-05T09:20:13.860Z
64c243175b329de1b1e6a048
https://cdn-avatars.huggingface.co/v1/production/uploads/64c243175b329de1b1e6a048/puCPybuNK1L62V5nRLttQ.jpeg
Zhihao Shi
user
somniis
66b03b6f06c013ef866bca2c
Juwei Lu
66b03b6f06c013ef866bca2d
Peng Dai
66b03b6f06c013ef866bca2e
Songcen Xu
66b03b6f06c013ef866bca2f
Li Cheng
66b03b6f06c013ef866bca30
Yee-Hong Yang
TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and
  Resampling
Given a 3D mesh, we aim to synthesize 3D textures that correspond to
arbitrary textual descriptions. Current methods for generating and assembling
textures from sampled views often result in prominent seams or excessive
smoothing. To tackle these issues, we present TexGen, a novel multi-view
sampling and resampling framework for texture generation leveraging a
pre-trained text-to-image diffusion model. For view consistent sampling, first
of all we maintain a texture map in RGB space that is parameterized by the
denoising step and updated after each sampling step of the diffusion model to
progressively reduce the view discrepancy. An attention-guided multi-view
sampling strategy is exploited to broadcast the appearance information across
views. To preserve texture details, we develop a noise resampling technique
that aids in the estimation of noise, generating inputs for subsequent
denoising steps, as directed by the text prompt and current texture map.
Through an extensive amount of qualitative and quantitative evaluations, we
demonstrate that our proposed method produces significantly better texture
quality for diverse 3D objects with a high degree of view consistency and rich
appearance details, outperforming current state-of-the-art methods.
Furthermore, our proposed texture generation technique can also be applied to
texture editing while preserving the original identity. More experimental
results are available at https://dong-huo.github.io/TexGen/
TexGen, a novel framework using a text-to-image diffusion model, generates high-quality, viewpoint-consistent 3D textures from textual descriptions, improving upon current methods.
multi-view sampling resampling framework text-to-image diffusion model denoising step attention-guided sampling noise resampling view consistency texture quality appearance details texture editing