Alchemist: Parametric Control of Material Properties with Diffusion
  Models
We propose a method to control material attributes of objects like roughness,
metallic, albedo, and transparency in real images. Our method capitalizes on
the generative prior of text-to-image models known for photorealism, employing
a scalar value and instructions to alter low-level material properties.
Addressing the lack of datasets with controlled material attributes, we
generated an object-centric synthetic dataset with physically-based materials.
Fine-tuning a modified pre-trained text-to-image model on this synthetic
dataset enables us to edit material properties in real-world images while
preserving all other attributes. We show the potential application of our model
to material edited NeRFs.
656fe1b5ec92f5a1a7300443
Prafull Sharma
656fe1b5ec92f5a1a7300444
Varun Jampani
admin_assigned
2023-12-06T12:53:15.826Z
630e7cfe3fc17ffc50f6e835
/avatars/0b742ff094a09f9374fafcd97ab9e002.svg
Varun Jampani
user
varunjampani
656fe1b5ec92f5a1a7300445
Yuanzhen Li
admin_assigned
2023-12-06T12:53:22.167Z
6334ee5831a2be3938c85639
/avatars/9821af26a585f54b595ec39f061f2250.svg
Yuanzhen Li
user
yzli
656fe1b5ec92f5a1a7300446
Xuhui Jia
admin_assigned
2023-12-06T12:53:32.191Z
634ce7ce05f736dff37aae5f
/avatars/1405c76a38f7ea497e4439f0e4e786a8.svg
Xuhui Jia
user
Jxh-cuit
656fe1b5ec92f5a1a7300447
Dmitry Lagun
656fe1b5ec92f5a1a7300448
Fredo Durand
656fe1b5ec92f5a1a7300449
William T. Freeman
admin_assigned
2023-12-06T12:54:22.457Z
654e866a85a5e608059a9b4f
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/O0T97EupMHH_iXcv1xYFr.png
William Freeman
user
mrpuppt
656fe1b5ec92f5a1a730044a
Mark Matthews
Alchemist: Parametric Control of Material Properties with Diffusion
  Models
We propose a method to control material attributes of objects like roughness,
metallic, albedo, and transparency in real images. Our method capitalizes on
the generative prior of text-to-image models known for photorealism, employing
a scalar value and instructions to alter low-level material properties.
Addressing the lack of datasets with controlled material attributes, we
generated an object-centric synthetic dataset with physically-based materials.
Fine-tuning a modified pre-trained text-to-image model on this synthetic
dataset enables us to edit material properties in real-world images while
preserving all other attributes. We show the potential application of our model
to material edited NeRFs.
A method to control material attributes in real images using text-to-image models fine-tuned on a synthetic dataset with physically-based materials.
generative prior text-to-image models synthetic dataset physically-based materials NeRFs