TextureDreamer: Image-guided Texture Synthesis through Geometry-aware
  Diffusion
We present TextureDreamer, a novel image-guided texture synthesis method to
transfer relightable textures from a small number of input images (3 to 5) to
target 3D shapes across arbitrary categories. Texture creation is a pivotal
challenge in vision and graphics. Industrial companies hire experienced artists
to manually craft textures for 3D assets. Classical methods require densely
sampled views and accurately aligned geometry, while learning-based methods are
confined to category-specific shapes within the dataset. In contrast,
TextureDreamer can transfer highly detailed, intricate textures from real-world
environments to arbitrary objects with only a few casually captured images,
potentially significantly democratizing texture creation. Our core idea,
personalized geometry-aware score distillation (PGSD), draws inspiration from
recent advancements in diffuse models, including personalized modeling for
texture information extraction, variational score distillation for detailed
appearance synthesis, and explicit geometry guidance with ControlNet. Our
integration and several essential modifications substantially improve the
texture quality. Experiments on real images spanning different categories show
that TextureDreamer can successfully transfer highly realistic, semantic
meaningful texture to arbitrary objects, surpassing the visual quality of
previous state-of-the-art.
65a8aa336cb12bdc272cc1f8
Yu-Ying Yeh
65a8aa336cb12bdc272cc1f9
Jia-Bin Huang
admin_assigned
2024-01-18T10:28:49.173Z
641c139b73296f7ee256970c
/avatars/5a2550d95e686640242840ad3bd0e680.svg
Jiabin Huang
user
YellowAddice
65a8aa336cb12bdc272cc1fa
Changil Kim
65a8aa336cb12bdc272cc1fb
Lei Xiao
65a8aa336cb12bdc272cc1fc
Thu Nguyen-Phuoc
admin_assigned
2024-01-18T10:29:23.236Z
6304b663c2f4f2d4929ad200
/avatars/805d4dab512fc53d8bd6bcb60db3821b.svg
Thu Nguyen-Phuoc
user
TNP23
65a8aa336cb12bdc272cc1fd
Numair Khan
65a8aa336cb12bdc272cc1fe
Cheng Zhang
65a8aa336cb12bdc272cc1ff
Manmohan Chandraker
65a8aa336cb12bdc272cc200
Carl S Marshall
65a8aa336cb12bdc272cc201
Zhao Dong
claimed_verified
2024-01-18T14:41:35.347Z
65a935e839cb91f1fa4c7507
/avatars/80a8c4f9a42d68c6981cd46c3d3562f2.svg
Zhao Dong
user
flycooler
65a8aa336cb12bdc272cc202
Zhengqin Li
admin_assigned
2024-01-18T10:31:49.305Z
64ff5b2569219ce3e47893f1
/avatars/36bfb3de85488c0efa75d741eac9c40e.svg
Li
user
Zhengqin
TextureDreamer: Image-guided Texture Synthesis through Geometry-aware
  Diffusion
We present TextureDreamer, a novel image-guided texture synthesis method to
transfer relightable textures from a small number of input images (3 to 5) to
target 3D shapes across arbitrary categories. Texture creation is a pivotal
challenge in vision and graphics. Industrial companies hire experienced artists
to manually craft textures for 3D assets. Classical methods require densely
sampled views and accurately aligned geometry, while learning-based methods are
confined to category-specific shapes within the dataset. In contrast,
TextureDreamer can transfer highly detailed, intricate textures from real-world
environments to arbitrary objects with only a few casually captured images,
potentially significantly democratizing texture creation. Our core idea,
personalized geometry-aware score distillation (PGSD), draws inspiration from
recent advancements in diffuse models, including personalized modeling for
texture information extraction, variational score distillation for detailed
appearance synthesis, and explicit geometry guidance with ControlNet. Our
integration and several essential modifications substantially improve the
texture quality. Experiments on real images spanning different categories show
that TextureDreamer can successfully transfer highly realistic, semantic
meaningful texture to arbitrary objects, surpassing the visual quality of
previous state-of-the-art.
TextureDreamer uses personalized geometry-aware score distillation to transfer detailed textures from a few images to arbitrary 3D shapes, surpassing existing methods.
TextureDreamer personalized geometry-aware score distillation diffuse models variational score distillation ControlNet texture synthesis