title: StoryMaker: Towards Holistic Consistent Characters in Text-to-image
    Generation
summary: Tuning-free personalized image generation methods have achieved significant
  success in maintaining facial consistency, i.e., identities, even with multiple
  characters. However, the lack of holistic consistency in scenes with multiple
  characters hampers these methods' ability to create a cohesive narrative. In
  this paper, we introduce StoryMaker, a personalization solution that preserves
  not only facial consistency but also clothing, hairstyles, and body
  consistency, thus facilitating the creation of a story through a series of
  images. StoryMaker incorporates conditions based on face identities and cropped
  character images, which include clothing, hairstyles, and bodies. Specifically,
  we integrate the facial identity information with the cropped character images
  using the Positional-aware Perceiver Resampler (PPR) to obtain distinct
  character features. To prevent intermingling of multiple characters and the
  background, we separately constrain the cross-attention impact regions of
  different characters and the background using MSE loss with segmentation masks.
  Additionally, we train the generation network conditioned on poses to promote
  decoupling from poses. A LoRA is also employed to enhance fidelity and quality.
  Experiments underscore the effectiveness of our approach. StoryMaker supports
  numerous applications and is compatible with other societal plug-ins. Our
  source codes and model weights are available at
  https://github.com/RedAIGC/StoryMaker.
paper_authors._id: 66ecd9e0fcc1c455f812a695
paper_authors.name: Zhengguang Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T08:55:24.898Z
paper_authors._id: 64b51874849c9e54e0d55f2d
paper_authors.avatarUrl: /avatars/07228522b73d591f40f4cd3794851752.svg
paper_authors.fullname: zhouzhengguang
paper_authors.type: user
paper_authors.user: huaquan
paper_authors._id: 66ecd9e0fcc1c455f812a696
paper_authors.name: Jing Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T08:55:31.419Z
paper_authors._id: 632c2a556bcb864974cc1558
paper_authors.avatarUrl: /avatars/000cd418f438a2754366b94dcf0aee10.svg
paper_authors.fullname: Jing Li
paper_authors.type: user
paper_authors.user: jingli
paper_authors._id: 66ecd9e0fcc1c455f812a697
paper_authors.name: Huaxia Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T08:55:37.541Z
paper_authors._id: 64b894cbe436bbca16627f4b
paper_authors.avatarUrl: /avatars/955908f51b2fbee9db8dd071d3f23d3c.svg
paper_authors.fullname: huaxiali
paper_authors.type: user
paper_authors.user: huaxiali
paper_authors._id: 66ecd9e0fcc1c455f812a698
paper_authors.name: Nemo Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T08:55:43.096Z
paper_authors._id: 64c87825cbcae9f106cf90b0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c87825cbcae9f106cf90b0/LAug686nylBUYojPLrkyb.png
paper_authors.fullname: Nemo CHEN
paper_authors.type: user
paper_authors.user: chenyibo89
paper_authors._id: 66ecd9e0fcc1c455f812a699
paper_authors.name: Xu Tang
paper_title: StoryMaker: Towards Holistic Consistent Characters in Text-to-image
    Generation
paper_summary: Tuning-free personalized image generation methods have achieved significant
  success in maintaining facial consistency, i.e., identities, even with multiple
  characters. However, the lack of holistic consistency in scenes with multiple
  characters hampers these methods' ability to create a cohesive narrative. In
  this paper, we introduce StoryMaker, a personalization solution that preserves
  not only facial consistency but also clothing, hairstyles, and body
  consistency, thus facilitating the creation of a story through a series of
  images. StoryMaker incorporates conditions based on face identities and cropped
  character images, which include clothing, hairstyles, and bodies. Specifically,
  we integrate the facial identity information with the cropped character images
  using the Positional-aware Perceiver Resampler (PPR) to obtain distinct
  character features. To prevent intermingling of multiple characters and the
  background, we separately constrain the cross-attention impact regions of
  different characters and the background using MSE loss with segmentation masks.
  Additionally, we train the generation network conditioned on poses to promote
  decoupling from poses. A LoRA is also employed to enhance fidelity and quality.
  Experiments underscore the effectiveness of our approach. StoryMaker supports
  numerous applications and is compatible with other societal plug-ins. Our
  source codes and model weights are available at
  https://github.com/RedAIGC/StoryMaker.
paper_ai_keywords: Positional-aware Perceiver Resampler (PPR) cross-attention MSE loss segmentation masks LoRA