title: CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via
    Adversarial Latent Search
summary: The success of deep learning based face recognition systems has given rise to
  serious privacy concerns due to their ability to enable unauthorized tracking
  of users in the digital world. Existing methods for enhancing privacy fail to
  generate naturalistic images that can protect facial privacy without
  compromising user experience. We propose a novel two-step approach for facial
  privacy protection that relies on finding adversarial latent codes in the
  low-dimensional manifold of a pretrained generative model. The first step
  inverts the given face image into the latent space and finetunes the generative
  model to achieve an accurate reconstruction of the given image from its latent
  code. This step produces a good initialization, aiding the generation of
  high-quality faces that resemble the given identity. Subsequently, user-defined
  makeup text prompts and identity-preserving regularization are used to guide
  the search for adversarial codes in the latent space. Extensive experiments
  demonstrate that faces generated by our approach have stronger black-box
  transferability with an absolute gain of 12.06% over the state-of-the-art
  facial privacy protection approach under the face verification task. Finally,
  we demonstrate the effectiveness of the proposed approach for commercial face
  recognition systems. Our code is available at
  https://github.com/fahadshamshad/Clip2Protect.
paper_authors._id: 64926adea820f46b2cc19ca4
paper_authors.name: Fahad Shamshad
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-21T09:16:53.153Z
paper_authors._id: 6492580a9f2bbd1d8ed70ca4
paper_authors.avatarUrl: /avatars/ee2be9258ff732e11f02317a2e61eab9.svg
paper_authors.fullname: Shamshad
paper_authors.type: user
paper_authors.user: Fahad786
paper_authors._id: 64926adea820f46b2cc19ca5
paper_authors.name: Muzammal Naseer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-23T14:28:05.603Z
paper_authors._id: 6495aae2e9b6d89a5c65cf15
paper_authors.avatarUrl: /avatars/c176d68af77698745c23da8b92424738.svg
paper_authors.fullname: Muzammal Naseer
paper_authors.type: user
paper_authors.user: muzammal
paper_authors._id: 64926adea820f46b2cc19ca6
paper_authors.name: Karthik Nandakumar
paper_title: CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via
    Adversarial Latent Search
paper_summary: The success of deep learning based face recognition systems has given rise to
  serious privacy concerns due to their ability to enable unauthorized tracking
  of users in the digital world. Existing methods for enhancing privacy fail to
  generate naturalistic images that can protect facial privacy without
  compromising user experience. We propose a novel two-step approach for facial
  privacy protection that relies on finding adversarial latent codes in the
  low-dimensional manifold of a pretrained generative model. The first step
  inverts the given face image into the latent space and finetunes the generative
  model to achieve an accurate reconstruction of the given image from its latent
  code. This step produces a good initialization, aiding the generation of
  high-quality faces that resemble the given identity. Subsequently, user-defined
  makeup text prompts and identity-preserving regularization are used to guide
  the search for adversarial codes in the latent space. Extensive experiments
  demonstrate that faces generated by our approach have stronger black-box
  transferability with an absolute gain of 12.06% over the state-of-the-art
  facial privacy protection approach under the face verification task. Finally,
  we demonstrate the effectiveness of the proposed approach for commercial face
  recognition systems. Our code is available at
  https://github.com/fahadshamshad/Clip2Protect.
paper_ai_summary: A novel two-step method uses adversarial latent codes in a pretrained generative model to enhance facial privacy while maintaining identity in face recognition systems.
paper_ai_keywords: adversarial latent codes low-dimensional manifold pretrained generative model latent space face verification task identity-preserving regularization