title: ZePo: Zero-Shot Portrait Stylization with Faster Sampling
summary: Diffusion-based text-to-image generation models have significantly advanced
  the field of art content synthesis. However, current portrait stylization
  methods generally require either model fine-tuning based on examples or the
  employment of DDIM Inversion to revert images to noise space, both of which
  substantially decelerate the image generation process. To overcome these
  limitations, this paper presents an inversion-free portrait stylization
  framework based on diffusion models that accomplishes content and style feature
  fusion in merely four sampling steps. We observed that Latent Consistency
  Models employing consistency distillation can effectively extract
  representative Consistency Features from noisy images. To blend the Consistency
  Features extracted from both content and style images, we introduce a Style
  Enhancement Attention Control technique that meticulously merges content and
  style features within the attention space of the target image. Moreover, we
  propose a feature merging strategy to amalgamate redundant features in
  Consistency Features, thereby reducing the computational load of attention
  control. Extensive experiments have validated the effectiveness of our proposed
  framework in enhancing stylization efficiency and fidelity. The code is
  available at https://github.com/liujin112/ZePo.
paper_authors._id: 66baf3a0a47daa804db162b3
paper_authors.name: Jin Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-13T07:37:53.062Z
paper_authors._id: 631afac4838c835180e725eb
paper_authors.avatarUrl: /avatars/51e6ec2cf79e04bb078f5ef804ebc394.svg
paper_authors.fullname: Jin Liu
paper_authors.type: user
paper_authors.user: Jinl
paper_authors._id: 66baf3a0a47daa804db162b4
paper_authors.name: Huaibo Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:08:01.710Z
paper_authors._id: 642259feeaad1bcb28b44f55
paper_authors.avatarUrl: /avatars/685791fc7e66d685e9446228c9e278b1.svg
paper_authors.fullname: Huaibo Huang
paper_authors.type: user
paper_authors.user: Bob072
paper_authors._id: 66baf3a0a47daa804db162b5
paper_authors.name: Jie Cao
paper_authors._id: 66baf3a0a47daa804db162b6
paper_authors.name: Ran He
paper_title: ZePo: Zero-Shot Portrait Stylization with Faster Sampling
paper_summary: Diffusion-based text-to-image generation models have significantly advanced
  the field of art content synthesis. However, current portrait stylization
  methods generally require either model fine-tuning based on examples or the
  employment of DDIM Inversion to revert images to noise space, both of which
  substantially decelerate the image generation process. To overcome these
  limitations, this paper presents an inversion-free portrait stylization
  framework based on diffusion models that accomplishes content and style feature
  fusion in merely four sampling steps. We observed that Latent Consistency
  Models employing consistency distillation can effectively extract
  representative Consistency Features from noisy images. To blend the Consistency
  Features extracted from both content and style images, we introduce a Style
  Enhancement Attention Control technique that meticulously merges content and
  style features within the attention space of the target image. Moreover, we
  propose a feature merging strategy to amalgamate redundant features in
  Consistency Features, thereby reducing the computational load of attention
  control. Extensive experiments have validated the effectiveness of our proposed
  framework in enhancing stylization efficiency and fidelity. The code is
  available at https://github.com/liujin112/ZePo.
paper_ai_summary: A diffusion-based framework for text-to-image portrait stylization enhances efficiency and fidelity without fine-tuning or DDIM inversion by using Latent Consistency Models and Style Enhancement Attention Control.
paper_ai_keywords: diffusion models portrait stylization DDIM Inversion Latent Consistency Models Consistency Features Style Enhancement Attention Control attention space feature merging strategy