title: PIA: Your Personalized Image Animator via Plug-and-Play Modules in
    Text-to-Image Models
summary: Recent advancements in personalized text-to-image (T2I) models have
  revolutionized content creation, empowering non-experts to generate stunning
  images with unique styles. While promising, adding realistic motions into these
  personalized images by text poses significant challenges in preserving distinct
  styles, high-fidelity details, and achieving motion controllability by text. In
  this paper, we present PIA, a Personalized Image Animator that excels in
  aligning with condition images, achieving motion controllability by text, and
  the compatibility with various personalized T2I models without specific tuning.
  To achieve these goals, PIA builds upon a base T2I model with well-trained
  temporal alignment layers, allowing for the seamless transformation of any
  personalized T2I model into an image animation model. A key component of PIA is
  the introduction of the condition module, which utilizes the condition frame
  and inter-frame affinity as input to transfer appearance information guided by
  the affinity hint for individual frame synthesis in the latent space. This
  design mitigates the challenges of appearance-related image alignment within
  and allows for a stronger focus on aligning with motion-related guidance.
paper_authors._id: 6584f9100f26133956091593
paper_authors.name: Yiming Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-02T20:08:34.819Z
paper_authors._id: 64ccaa4687ec96aa4752e754
paper_authors.avatarUrl: /avatars/d2dd2040a521de4f55c7335cb7771c75.svg
paper_authors.fullname: Yiming Zhang
paper_authors.type: user
paper_authors.user: ymzhang319
paper_authors._id: 6584f9100f26133956091594
paper_authors.name: Zhening Xing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T12:30:04.341Z
paper_authors._id: 62fb2a9dc95d426ff8f74c8d
paper_authors.avatarUrl: /avatars/25c1a68ee7b7d0cc7e9f56bde37f4914.svg
paper_authors.fullname: Zhening Xing
paper_authors.type: user
paper_authors.user: Leoxing
paper_authors._id: 6584f9100f26133956091595
paper_authors.name: Yanhong Zeng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-22T13:00:39.833Z
paper_authors._id: 63d4b843df01ef426a0f79fb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676365795587-63d4b843df01ef426a0f79fb.jpeg
paper_authors.fullname: Yanhong Zeng
paper_authors.type: user
paper_authors.user: zengyh1900
paper_authors._id: 6584f9100f26133956091596
paper_authors.name: Youqing Fang
paper_authors._id: 6584f9100f26133956091597
paper_authors.name: Kai Chen
paper_title: PIA: Your Personalized Image Animator via Plug-and-Play Modules in
    Text-to-Image Models
paper_summary: Recent advancements in personalized text-to-image (T2I) models have
  revolutionized content creation, empowering non-experts to generate stunning
  images with unique styles. While promising, adding realistic motions into these
  personalized images by text poses significant challenges in preserving distinct
  styles, high-fidelity details, and achieving motion controllability by text. In
  this paper, we present PIA, a Personalized Image Animator that excels in
  aligning with condition images, achieving motion controllability by text, and
  the compatibility with various personalized T2I models without specific tuning.
  To achieve these goals, PIA builds upon a base T2I model with well-trained
  temporal alignment layers, allowing for the seamless transformation of any
  personalized T2I model into an image animation model. A key component of PIA is
  the introduction of the condition module, which utilizes the condition frame
  and inter-frame affinity as input to transfer appearance information guided by
  the affinity hint for individual frame synthesis in the latent space. This
  design mitigates the challenges of appearance-related image alignment within
  and allows for a stronger focus on aligning with motion-related guidance.
paper_ai_summary: PIA enhances personalized T2I models to generate animations by integrating temporal alignment and condition modules for motion control and appearance consistency.
paper_ai_keywords: personalized T2I models temporal alignment layers condition module condition frame inter-frame affinity latent space image animation model motion controllability appearance-related image alignment