title: Alchemist: Parametric Control of Material Properties with Diffusion
    Models
summary: We propose a method to control material attributes of objects like roughness,
  metallic, albedo, and transparency in real images. Our method capitalizes on
  the generative prior of text-to-image models known for photorealism, employing
  a scalar value and instructions to alter low-level material properties.
  Addressing the lack of datasets with controlled material attributes, we
  generated an object-centric synthetic dataset with physically-based materials.
  Fine-tuning a modified pre-trained text-to-image model on this synthetic
  dataset enables us to edit material properties in real-world images while
  preserving all other attributes. We show the potential application of our model
  to material edited NeRFs.
paper_authors._id: 656fe1b5ec92f5a1a7300443
paper_authors.name: Prafull Sharma
paper_authors._id: 656fe1b5ec92f5a1a7300444
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:53:15.826Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_authors._id: 656fe1b5ec92f5a1a7300445
paper_authors.name: Yuanzhen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:53:22.167Z
paper_authors._id: 6334ee5831a2be3938c85639
paper_authors.avatarUrl: /avatars/9821af26a585f54b595ec39f061f2250.svg
paper_authors.fullname: Yuanzhen Li
paper_authors.type: user
paper_authors.user: yzli
paper_authors._id: 656fe1b5ec92f5a1a7300446
paper_authors.name: Xuhui Jia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:53:32.191Z
paper_authors._id: 634ce7ce05f736dff37aae5f
paper_authors.avatarUrl: /avatars/1405c76a38f7ea497e4439f0e4e786a8.svg
paper_authors.fullname: Xuhui Jia
paper_authors.type: user
paper_authors.user: Jxh-cuit
paper_authors._id: 656fe1b5ec92f5a1a7300447
paper_authors.name: Dmitry Lagun
paper_authors._id: 656fe1b5ec92f5a1a7300448
paper_authors.name: Fredo Durand
paper_authors._id: 656fe1b5ec92f5a1a7300449
paper_authors.name: William T. Freeman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-06T12:54:22.457Z
paper_authors._id: 654e866a85a5e608059a9b4f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/O0T97EupMHH_iXcv1xYFr.png
paper_authors.fullname: William Freeman
paper_authors.type: user
paper_authors.user: mrpuppt
paper_authors._id: 656fe1b5ec92f5a1a730044a
paper_authors.name: Mark Matthews
paper_title: Alchemist: Parametric Control of Material Properties with Diffusion
    Models
paper_summary: We propose a method to control material attributes of objects like roughness,
  metallic, albedo, and transparency in real images. Our method capitalizes on
  the generative prior of text-to-image models known for photorealism, employing
  a scalar value and instructions to alter low-level material properties.
  Addressing the lack of datasets with controlled material attributes, we
  generated an object-centric synthetic dataset with physically-based materials.
  Fine-tuning a modified pre-trained text-to-image model on this synthetic
  dataset enables us to edit material properties in real-world images while
  preserving all other attributes. We show the potential application of our model
  to material edited NeRFs.
paper_ai_summary: A method to control material attributes in real images using text-to-image models fine-tuned on a synthetic dataset with physically-based materials.
paper_ai_keywords: generative prior text-to-image models synthetic dataset physically-based materials NeRFs