Figure 2. Overview of our method during inference. Please note that we use a latent diffusion model [57], yet we illustrate the denoising
process in the original image space for visualization purposes. We perform standard 3DMM ﬁtting to get a partial UV texture via image-
to-uv rasterization. Then, starting from random noise, we utilize the known texture to guide the sampling process of a texture/reﬂectance
diffusion model towards completing the unobserved pixels. Each denoising step, from zt to zt−1 (t ∈ {1, . . . , T}), follows an inpainting
approach similar to MCG [11] (see Eq. 9): 1). The reﬂectance maps and unobserved texture pixels are updated based on reverse diffusion
sampling and manifold constraints, while 2) the known pixels are directly sampled from the input texture via forward diffusion ((cid:12) and
⊕ denote the Hadamard product and addition respectively). Note that masking is only applied to the texture, while the reﬂectance maps
Relightify-Relightable3DFacesfromaSingleImageviaDiffusionModels