CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via
  Adversarial Latent Search
The success of deep learning based face recognition systems has given rise to
serious privacy concerns due to their ability to enable unauthorized tracking
of users in the digital world. Existing methods for enhancing privacy fail to
generate naturalistic images that can protect facial privacy without
compromising user experience. We propose a novel two-step approach for facial
privacy protection that relies on finding adversarial latent codes in the
low-dimensional manifold of a pretrained generative model. The first step
inverts the given face image into the latent space and finetunes the generative
model to achieve an accurate reconstruction of the given image from its latent
code. This step produces a good initialization, aiding the generation of
high-quality faces that resemble the given identity. Subsequently, user-defined
makeup text prompts and identity-preserving regularization are used to guide
the search for adversarial codes in the latent space. Extensive experiments
demonstrate that faces generated by our approach have stronger black-box
transferability with an absolute gain of 12.06% over the state-of-the-art
facial privacy protection approach under the face verification task. Finally,
we demonstrate the effectiveness of the proposed approach for commercial face
recognition systems. Our code is available at
https://github.com/fahadshamshad/Clip2Protect.
64926adea820f46b2cc19ca4
Fahad Shamshad
claimed_verified
2023-06-21T09:16:53.153Z
6492580a9f2bbd1d8ed70ca4
/avatars/ee2be9258ff732e11f02317a2e61eab9.svg
Shamshad
user
Fahad786
64926adea820f46b2cc19ca5
Muzammal Naseer
claimed_verified
2023-06-23T14:28:05.603Z
6495aae2e9b6d89a5c65cf15
/avatars/c176d68af77698745c23da8b92424738.svg
Muzammal Naseer
user
muzammal
64926adea820f46b2cc19ca6
Karthik Nandakumar
CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via
  Adversarial Latent Search
The success of deep learning based face recognition systems has given rise to
serious privacy concerns due to their ability to enable unauthorized tracking
of users in the digital world. Existing methods for enhancing privacy fail to
generate naturalistic images that can protect facial privacy without
compromising user experience. We propose a novel two-step approach for facial
privacy protection that relies on finding adversarial latent codes in the
low-dimensional manifold of a pretrained generative model. The first step
inverts the given face image into the latent space and finetunes the generative
model to achieve an accurate reconstruction of the given image from its latent
code. This step produces a good initialization, aiding the generation of
high-quality faces that resemble the given identity. Subsequently, user-defined
makeup text prompts and identity-preserving regularization are used to guide
the search for adversarial codes in the latent space. Extensive experiments
demonstrate that faces generated by our approach have stronger black-box
transferability with an absolute gain of 12.06% over the state-of-the-art
facial privacy protection approach under the face verification task. Finally,
we demonstrate the effectiveness of the proposed approach for commercial face
recognition systems. Our code is available at
https://github.com/fahadshamshad/Clip2Protect.
A novel two-step method uses adversarial latent codes in a pretrained generative model to enhance facial privacy while maintaining identity in face recognition systems.
adversarial latent codes low-dimensional manifold pretrained generative model latent space face verification task identity-preserving regularization