title: TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion
    Models
summary: We present TexFusion (Texture Diffusion), a new method to synthesize textures
  for given 3D geometries, using large-scale text-guided image diffusion models.
  In contrast to recent works that leverage 2D text-to-image diffusion models to
  distill 3D objects using a slow and fragile optimization process, TexFusion
  introduces a new 3D-consistent generation technique specifically designed for
  texture synthesis that employs regular diffusion model sampling on different 2D
  rendered views. Specifically, we leverage latent diffusion models, apply the
  diffusion model's denoiser on a set of 2D renders of the 3D object, and
  aggregate the different denoising predictions on a shared latent texture map.
  Final output RGB textures are produced by optimizing an intermediate neural
  color field on the decodings of 2D renders of the latent texture. We thoroughly
  validate TexFusion and show that we can efficiently generate diverse, high
  quality and globally coherent textures. We achieve state-of-the-art text-guided
  texture synthesis performance using only image diffusion models, while avoiding
  the pitfalls of previous distillation-based methods. The text-conditioning
  offers detailed control and we also do not rely on any ground truth 3D textures
  for training. This makes our method versatile and applicable to a broad range
  of geometry and texture types. We hope that TexFusion will advance AI-based
  texturing of 3D assets for applications in virtual reality, game design,
  simulation, and more.
paper_authors._id: 6537dc0146cdab563dbdebd1
paper_authors.name: Tianshi Cao
paper_authors._id: 6537dc0146cdab563dbdebd2
paper_authors.name: Karsten Kreis
paper_authors._id: 6537dc0146cdab563dbdebd3
paper_authors.name: Sanja Fidler
paper_authors._id: 6537dc0146cdab563dbdebd4
paper_authors.name: Nicholas Sharp
paper_authors._id: 6537dc0146cdab563dbdebd5
paper_authors.name: Kangxue Yin
paper_title: TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion
    Models
paper_summary: We present TexFusion (Texture Diffusion), a new method to synthesize textures
  for given 3D geometries, using large-scale text-guided image diffusion models.
  In contrast to recent works that leverage 2D text-to-image diffusion models to
  distill 3D objects using a slow and fragile optimization process, TexFusion
  introduces a new 3D-consistent generation technique specifically designed for
  texture synthesis that employs regular diffusion model sampling on different 2D
  rendered views. Specifically, we leverage latent diffusion models, apply the
  diffusion model's denoiser on a set of 2D renders of the 3D object, and
  aggregate the different denoising predictions on a shared latent texture map.
  Final output RGB textures are produced by optimizing an intermediate neural
  color field on the decodings of 2D renders of the latent texture. We thoroughly
  validate TexFusion and show that we can efficiently generate diverse, high
  quality and globally coherent textures. We achieve state-of-the-art text-guided
  texture synthesis performance using only image diffusion models, while avoiding
  the pitfalls of previous distillation-based methods. The text-conditioning
  offers detailed control and we also do not rely on any ground truth 3D textures
  for training. This makes our method versatile and applicable to a broad range
  of geometry and texture types. We hope that TexFusion will advance AI-based
  texturing of 3D assets for applications in virtual reality, game design,
  simulation, and more.
paper_ai_keywords: texture diffusion text-guided image diffusion models 3D-consistent generation latent diffusion models diffusion model's denoiser latent texture map neural color field