title: Jointly Generating Multi-view Consistent PBR Textures using
    Collaborative Control
summary: Multi-view consistency remains a challenge for image diffusion models. Even
  within the Text-to-Texture problem, where perfect geometric correspondences are
  known a priori, many methods fail to yield aligned predictions across views,
  necessitating non-trivial fusion methods to incorporate the results onto the
  original mesh. We explore this issue for a Collaborative Control workflow
  specifically in PBR Text-to-Texture. Collaborative Control directly models PBR
  image probability distributions, including normal bump maps; to our knowledge,
  the only diffusion model to directly output full PBR stacks. We discuss the
  design decisions involved in making this model multi-view consistent, and
  demonstrate the effectiveness of our approach in ablation studies, as well as
  practical applications.
paper_authors._id: 67079b9754069379bd86c10c
paper_authors.name: Shimon Vainer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:46:53.809Z
paper_authors._id: 63f9e95db31627d24009de1a
paper_authors.avatarUrl: /avatars/53e6782c38bd1babca03389ae15a7d2e.svg
paper_authors.fullname: Shimon Vainer
paper_authors.type: user
paper_authors.user: esx2ve
paper_authors._id: 67079b9754069379bd86c10d
paper_authors.name: Konstantin Kutsy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:46:59.437Z
paper_authors._id: 64636800d4d34b01f45cc40d
paper_authors.avatarUrl: /avatars/f423e37b8461f3bda11021b3abf80ee1.svg
paper_authors.fullname: Konstantin Kutsy
paper_authors.type: user
paper_authors.user: bostadynamics
paper_authors._id: 67079b9754069379bd86c10e
paper_authors.name: Dante De Nigris
paper_authors._id: 67079b9754069379bd86c10f
paper_authors.name: Ciara Rowles
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T09:29:55.515Z
paper_authors._id: 63357214eb6132ca653020e7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63357214eb6132ca653020e7/A_imoyFDx30wgi0guG_s8.png
paper_authors.fullname: Ciara
paper_authors.type: user
paper_authors.user: CiaraRowles
paper_authors._id: 67079b9754069379bd86c110
paper_authors.name: Slava Elizarov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:47:07.631Z
paper_authors._id: 639b56244bbcd176745a6e6d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639b56244bbcd176745a6e6d/a1DRanMtKUNJNcrt3ZmOg.jpeg
paper_authors.fullname: Slava Elizarov
paper_authors.type: user
paper_authors.user: SlavaElizarov
paper_authors._id: 67079b9754069379bd86c111
paper_authors.name: Simon Donné
paper_title: Jointly Generating Multi-view Consistent PBR Textures using
    Collaborative Control
paper_summary: Multi-view consistency remains a challenge for image diffusion models. Even
  within the Text-to-Texture problem, where perfect geometric correspondences are
  known a priori, many methods fail to yield aligned predictions across views,
  necessitating non-trivial fusion methods to incorporate the results onto the
  original mesh. We explore this issue for a Collaborative Control workflow
  specifically in PBR Text-to-Texture. Collaborative Control directly models PBR
  image probability distributions, including normal bump maps; to our knowledge,
  the only diffusion model to directly output full PBR stacks. We discuss the
  design decisions involved in making this model multi-view consistent, and
  demonstrate the effectiveness of our approach in ablation studies, as well as
  practical applications.
paper_ai_keywords: diffusion models Text-to-Texture perfect geometric correspondences multi-view consistency fusion methods Collaborative Control PBR Text-to-Texture PBR image probability distributions normal bump maps full PBR stacks ablation studies