  The multitude of makeup products available can make it challenging to find
the ideal match for desired attributes. An intelligent approach for product
discovery is required to enhance the makeup shopping experience to make it more
convenient and satisfying. However, enabling accurate and efficient product
discovery requires extracting detailed attributes like color and finish type.
Our work introduces an automated pipeline that utilizes multiple customized
machine learning models to extract essential material attributes from makeup
product images. Our pipeline is versatile and capable of handling various
makeup products. To showcase the efficacy of our pipeline, we conduct extensive
experiments on eyeshadow products (both single and multi-shade ones), a
challenging makeup product known for its diverse range of shapes, colors, and
finish types. Furthermore, we demonstrate the applicability of our approach by
successfully extending it to other makeup categories like lipstick and
foundation, showcasing its adaptability and effectiveness across different
beauty products. Additionally, we conduct ablation experiments to demonstrate
the superiority of our machine learning pipeline over human labeling methods in
terms of reliability. Our proposed method showcases its effectiveness in
cross-category product discovery, specifically in recommending makeup products
that perfectly match a specified outfit. Lastly, we also demonstrate the
application of these material attributes in enabling virtual-try-on experiences
which makes makeup shopping experience significantly more engaging.

Computer Vision and Pattern Recognition;Artificial Intelligence