title: Training-Free Consistent Text-to-Image Generation
summary: Text-to-image models offer a new level of creative flexibility by allowing
  users to guide the image generation process through natural language. However,
  using these models to consistently portray the same subject across diverse
  prompts remains challenging. Existing approaches fine-tune the model to teach
  it new words that describe specific user-provided subjects or add image
  conditioning to the model. These methods require lengthy per-subject
  optimization or large-scale pre-training. Moreover, they struggle to align
  generated images with text prompts and face difficulties in portraying multiple
  subjects. Here, we present ConsiStory, a training-free approach that enables
  consistent subject generation by sharing the internal activations of the
  pretrained model. We introduce a subject-driven shared attention block and
  correspondence-based feature injection to promote subject consistency between
  images. Additionally, we develop strategies to encourage layout diversity while
  maintaining subject consistency. We compare ConsiStory to a range of baselines,
  and demonstrate state-of-the-art performance on subject consistency and text
  alignment, without requiring a single optimization step. Finally, ConsiStory
  can naturally extend to multi-subject scenarios, and even enable training-free
  personalization for common objects.
paper_authors._id: 65c1a47f35cfc292c0c3d730
paper_authors.name: Yoad Tewel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:52:46.347Z
paper_authors._id: 6201838f305742efd5a6a0e3
paper_authors.avatarUrl: /avatars/4e47ab9ef1a4b9f59756a6d8c90a970f.svg
paper_authors.fullname: Yoad Tewel
paper_authors.type: user
paper_authors.user: YoadTew
paper_authors._id: 65c1a47f35cfc292c0c3d731
paper_authors.name: Omri Kaduri
paper_authors._id: 65c1a47f35cfc292c0c3d732
paper_authors.name: Rinon Gal
paper_authors._id: 65c1a47f35cfc292c0c3d733
paper_authors.name: Yoni Kasten
paper_authors._id: 65c1a47f35cfc292c0c3d734
paper_authors.name: Lior Wolf
paper_authors._id: 65c1a47f35cfc292c0c3d735
paper_authors.name: Gal Chechik
paper_authors._id: 65c1a47f35cfc292c0c3d736
paper_authors.name: Yuval Atzmon
paper_title: Training-Free Consistent Text-to-Image Generation
paper_summary: Text-to-image models offer a new level of creative flexibility by allowing
  users to guide the image generation process through natural language. However,
  using these models to consistently portray the same subject across diverse
  prompts remains challenging. Existing approaches fine-tune the model to teach
  it new words that describe specific user-provided subjects or add image
  conditioning to the model. These methods require lengthy per-subject
  optimization or large-scale pre-training. Moreover, they struggle to align
  generated images with text prompts and face difficulties in portraying multiple
  subjects. Here, we present ConsiStory, a training-free approach that enables
  consistent subject generation by sharing the internal activations of the
  pretrained model. We introduce a subject-driven shared attention block and
  correspondence-based feature injection to promote subject consistency between
  images. Additionally, we develop strategies to encourage layout diversity while
  maintaining subject consistency. We compare ConsiStory to a range of baselines,
  and demonstrate state-of-the-art performance on subject consistency and text
  alignment, without requiring a single optimization step. Finally, ConsiStory
  can naturally extend to multi-subject scenarios, and even enable training-free
  personalization for common objects.
paper_ai_summary: ConsiStory achieves state-of-the-art text-to-image subject consistency without fine-tuning by using shared internal activations, attention blocks, and feature injection.
paper_ai_keywords: text-to-image models subject consistency shared attention block correspondence-based feature injection pretrained model internal activations