FlashTex: Fast Relightable Mesh Texturing with LightControlNet
Manually creating textures for 3D meshes is time-consuming, even for expert
visual content creators. We propose a fast approach for automatically texturing
an input 3D mesh based on a user-provided text prompt. Importantly, our
approach disentangles lighting from surface material/reflectance in the
resulting texture so that the mesh can be properly relit and rendered in any
lighting environment. We introduce LightControlNet, a new text-to-image model
based on the ControlNet architecture, which allows the specification of the
desired lighting as a conditioning image to the model. Our text-to-texture
pipeline then constructs the texture in two stages. The first stage produces a
sparse set of visually consistent reference views of the mesh using
LightControlNet. The second stage applies a texture optimization based on Score
Distillation Sampling (SDS) that works with LightControlNet to increase the
texture quality while disentangling surface material from lighting. Our
pipeline is significantly faster than previous text-to-texture methods, while
producing high-quality and relightable textures.
65d586cfb17d8c0681f3998c
Kangle Deng
claimed_verified
2025-03-21T11:51:19.021Z
645d34ecce72244df7b29317
/avatars/1248933d9f89a15e67086325a8322d5e.svg
Kangle Deng
user
kangled
65d586cfb17d8c0681f3998d
Timothy Omernick
65d586cfb17d8c0681f3998e
Alexander Weiss
65d586cfb17d8c0681f3998f
Deva Ramanan
admin_assigned
2024-02-21T10:34:53.192Z
6337151b0267ebcf02640eb6
/avatars/14a723cafc5587043bdfb19304fc202d.svg
Deva Ramanan
user
devakramanan
65d586cfb17d8c0681f39990
Jun-Yan Zhu
admin_assigned
2024-02-21T10:35:22.450Z
63a0acc32fabbbb899952a2b
https://cdn-avatars.huggingface.co/v1/production/uploads/1671474335794-noauth.jpeg
Jun-Yan Zhu
user
junyanz
65d586cfb17d8c0681f39991
Tinghui Zhou
65d586cfb17d8c0681f39992
Maneesh Agrawala
admin_assigned
2024-02-21T10:36:41.426Z
642bb3e44f379cdc3deb26cd
/avatars/556e792f013bcbe3748d7c9c719070d8.svg
Maneesh Agrawala
user
magrawala
FlashTex: Fast Relightable Mesh Texturing with LightControlNet
Manually creating textures for 3D meshes is time-consuming, even for expert
visual content creators. We propose a fast approach for automatically texturing
an input 3D mesh based on a user-provided text prompt. Importantly, our
approach disentangles lighting from surface material/reflectance in the
resulting texture so that the mesh can be properly relit and rendered in any
lighting environment. We introduce LightControlNet, a new text-to-image model
based on the ControlNet architecture, which allows the specification of the
desired lighting as a conditioning image to the model. Our text-to-texture
pipeline then constructs the texture in two stages. The first stage produces a
sparse set of visually consistent reference views of the mesh using
LightControlNet. The second stage applies a texture optimization based on Score
Distillation Sampling (SDS) that works with LightControlNet to increase the
texture quality while disentangling surface material from lighting. Our
pipeline is significantly faster than previous text-to-texture methods, while
producing high-quality and relightable textures.
A LightControlNet-based pipeline automatically generates high-quality, relightable textures for 3D meshes from text prompts using a two-stage process.
LightControlNet ControlNet Score Distillation Sampling (SDS) text-to-image model text-to-texture reference views surface material lighting disentanglement