title: TextureDreamer: Image-guided Texture Synthesis through Geometry-aware
    Diffusion
summary: We present TextureDreamer, a novel image-guided texture synthesis method to
  transfer relightable textures from a small number of input images (3 to 5) to
  target 3D shapes across arbitrary categories. Texture creation is a pivotal
  challenge in vision and graphics. Industrial companies hire experienced artists
  to manually craft textures for 3D assets. Classical methods require densely
  sampled views and accurately aligned geometry, while learning-based methods are
  confined to category-specific shapes within the dataset. In contrast,
  TextureDreamer can transfer highly detailed, intricate textures from real-world
  environments to arbitrary objects with only a few casually captured images,
  potentially significantly democratizing texture creation. Our core idea,
  personalized geometry-aware score distillation (PGSD), draws inspiration from
  recent advancements in diffuse models, including personalized modeling for
  texture information extraction, variational score distillation for detailed
  appearance synthesis, and explicit geometry guidance with ControlNet. Our
  integration and several essential modifications substantially improve the
  texture quality. Experiments on real images spanning different categories show
  that TextureDreamer can successfully transfer highly realistic, semantic
  meaningful texture to arbitrary objects, surpassing the visual quality of
  previous state-of-the-art.
paper_authors._id: 65a8aa336cb12bdc272cc1f8
paper_authors.name: Yu-Ying Yeh
paper_authors._id: 65a8aa336cb12bdc272cc1f9
paper_authors.name: Jia-Bin Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T10:28:49.173Z
paper_authors._id: 641c139b73296f7ee256970c
paper_authors.avatarUrl: /avatars/5a2550d95e686640242840ad3bd0e680.svg
paper_authors.fullname: Jiabin Huang
paper_authors.type: user
paper_authors.user: YellowAddice
paper_authors._id: 65a8aa336cb12bdc272cc1fa
paper_authors.name: Changil Kim
paper_authors._id: 65a8aa336cb12bdc272cc1fb
paper_authors.name: Lei Xiao
paper_authors._id: 65a8aa336cb12bdc272cc1fc
paper_authors.name: Thu Nguyen-Phuoc
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T10:29:23.236Z
paper_authors._id: 6304b663c2f4f2d4929ad200
paper_authors.avatarUrl: /avatars/805d4dab512fc53d8bd6bcb60db3821b.svg
paper_authors.fullname: Thu Nguyen-Phuoc
paper_authors.type: user
paper_authors.user: TNP23
paper_authors._id: 65a8aa336cb12bdc272cc1fd
paper_authors.name: Numair Khan
paper_authors._id: 65a8aa336cb12bdc272cc1fe
paper_authors.name: Cheng Zhang
paper_authors._id: 65a8aa336cb12bdc272cc1ff
paper_authors.name: Manmohan Chandraker
paper_authors._id: 65a8aa336cb12bdc272cc200
paper_authors.name: Carl S Marshall
paper_authors._id: 65a8aa336cb12bdc272cc201
paper_authors.name: Zhao Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-18T14:41:35.347Z
paper_authors._id: 65a935e839cb91f1fa4c7507
paper_authors.avatarUrl: /avatars/80a8c4f9a42d68c6981cd46c3d3562f2.svg
paper_authors.fullname: Zhao Dong
paper_authors.type: user
paper_authors.user: flycooler
paper_authors._id: 65a8aa336cb12bdc272cc202
paper_authors.name: Zhengqin Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-18T10:31:49.305Z
paper_authors._id: 64ff5b2569219ce3e47893f1
paper_authors.avatarUrl: /avatars/36bfb3de85488c0efa75d741eac9c40e.svg
paper_authors.fullname: Li
paper_authors.type: user
paper_authors.user: Zhengqin
paper_title: TextureDreamer: Image-guided Texture Synthesis through Geometry-aware
    Diffusion
paper_summary: We present TextureDreamer, a novel image-guided texture synthesis method to
  transfer relightable textures from a small number of input images (3 to 5) to
  target 3D shapes across arbitrary categories. Texture creation is a pivotal
  challenge in vision and graphics. Industrial companies hire experienced artists
  to manually craft textures for 3D assets. Classical methods require densely
  sampled views and accurately aligned geometry, while learning-based methods are
  confined to category-specific shapes within the dataset. In contrast,
  TextureDreamer can transfer highly detailed, intricate textures from real-world
  environments to arbitrary objects with only a few casually captured images,
  potentially significantly democratizing texture creation. Our core idea,
  personalized geometry-aware score distillation (PGSD), draws inspiration from
  recent advancements in diffuse models, including personalized modeling for
  texture information extraction, variational score distillation for detailed
  appearance synthesis, and explicit geometry guidance with ControlNet. Our
  integration and several essential modifications substantially improve the
  texture quality. Experiments on real images spanning different categories show
  that TextureDreamer can successfully transfer highly realistic, semantic
  meaningful texture to arbitrary objects, surpassing the visual quality of
  previous state-of-the-art.
paper_ai_keywords: personalized geometry-aware score distillation (PGSD) diffuse models texture information extraction variational score distillation ControlNet geometry guidance inference semantic meaningful texture