  Facial makeup enriches the beauty of not only real humans but also virtual
characters; therefore, makeup for 3D facial models is highly in demand in
productions. However, painting directly on 3D faces and capturing real-world
makeup are costly, and extracting makeup from 2D images often struggles with
shading effects and occlusions. This paper presents the first method for
extracting makeup for 3D facial models from a single makeup portrait. Our
method consists of the following three steps. First, we exploit the strong
prior of 3D morphable models via regression-based inverse rendering to extract
coarse materials such as geometry and diffuse/specular albedos that are
represented in the UV space. Second, we refine the coarse materials, which may
have missing pixels due to occlusions. We apply inpainting and optimization.
Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse
albedo. Our method offers various applications for not only 3D facial models
but also 2D portrait images. The extracted makeup is well-aligned in the UV
space, from which we build a large-scale makeup dataset and a parametric makeup
model for 3D faces. Our disentangled materials also yield robust makeup
transfer and illumination-aware makeup interpolation/removal without a
reference image.

Computer Vision and Pattern Recognition;Graphics