title: Story-Adapter: A Training-free Iterative Framework for Long Story
    Visualization
summary: Story visualization, the task of generating coherent images based on a
  narrative, has seen significant advancements with the emergence of
  text-to-image models, particularly diffusion models. However, maintaining
  semantic consistency, generating high-quality fine-grained interactions, and
  ensuring computational feasibility remain challenging, especially in long story
  visualization (i.e., up to 100 frames). In this work, we propose a
  training-free and computationally efficient framework, termed Story-Adapter, to
  enhance the generative capability of long stories. Specifically, we propose an
  iterative paradigm to refine each generated image, leveraging both the text
  prompt and all generated images from the previous iteration. Central to our
  framework is a training-free global reference cross-attention module, which
  aggregates all generated images from the previous iteration to preserve
  semantic consistency across the entire story, while minimizing computational
  costs with global embeddings. This iterative process progressively optimizes
  image generation by repeatedly incorporating text constraints, resulting in
  more precise and fine-grained interactions. Extensive experiments validate the
  superiority of Story-Adapter in improving both semantic consistency and
  generative capability for fine-grained interactions, particularly in long story
  scenarios. The project page and associated code can be accessed via
  https://jwmao1.github.io/storyadapter .
paper_authors._id: 670745bbd6f7382cc428eafc
paper_authors.name: Jiawei Mao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:01:11.990Z
paper_authors._id: 65f261116c60cd168b05433c
paper_authors.avatarUrl: /avatars/0515bef9df2514707cbd3fc281891e2d.svg
paper_authors.fullname: Jiawei Mao
paper_authors.type: user
paper_authors.user: JohnWeck
paper_authors._id: 670745bbd6f7382cc428eafd
paper_authors.name: Xiaoke Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:01:17.167Z
paper_authors._id: 63318b2349a9563915469f3b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63318b2349a9563915469f3b/zlbeB2997i8YkoyOTb9FL.jpeg
paper_authors.fullname: Xiaoke Huang
paper_authors.type: user
paper_authors.user: xk-huang
paper_authors._id: 670745bbd6f7382cc428eafe
paper_authors.name: Yunfei Xie
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T07:44:16.578Z
paper_authors._id: 65f6afa36ea40b9a29dd45ca
paper_authors.avatarUrl: /avatars/a80a25fa70a89a3bf3d50ea2866f0df1.svg
paper_authors.fullname: Yunfei Xie
paper_authors.type: user
paper_authors.user: yunfeixie
paper_authors._id: 670745bbd6f7382cc428eaff
paper_authors.name: Yuanqi Chang
paper_authors._id: 670745bbd6f7382cc428eb00
paper_authors.name: Mude Hui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:01:57.443Z
paper_authors._id: 659f6f0de06dc8fc0e411224
paper_authors.avatarUrl: /avatars/e16170f36b949c8697f277f6b52c51c3.svg
paper_authors.fullname: MudeHui
paper_authors.type: user
paper_authors.user: MudeHui
paper_authors._id: 670745bbd6f7382cc428eb01
paper_authors.name: Bingjie Xu
paper_authors._id: 670745bbd6f7382cc428eb02
paper_authors.name: Yuyin Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:02:11.011Z
paper_authors._id: 66c7fb4ce2c92fe5b132f314
paper_authors.avatarUrl: /avatars/22d915fa339a70803c5c748255250256.svg
paper_authors.fullname: Yuyin Zhou
paper_authors.type: user
paper_authors.user: RitaCoding
paper_title: Story-Adapter: A Training-free Iterative Framework for Long Story
    Visualization
paper_summary: Story visualization, the task of generating coherent images based on a
  narrative, has seen significant advancements with the emergence of
  text-to-image models, particularly diffusion models. However, maintaining
  semantic consistency, generating high-quality fine-grained interactions, and
  ensuring computational feasibility remain challenging, especially in long story
  visualization (i.e., up to 100 frames). In this work, we propose a
  training-free and computationally efficient framework, termed Story-Adapter, to
  enhance the generative capability of long stories. Specifically, we propose an
  iterative paradigm to refine each generated image, leveraging both the text
  prompt and all generated images from the previous iteration. Central to our
  framework is a training-free global reference cross-attention module, which
  aggregates all generated images from the previous iteration to preserve
  semantic consistency across the entire story, while minimizing computational
  costs with global embeddings. This iterative process progressively optimizes
  image generation by repeatedly incorporating text constraints, resulting in
  more precise and fine-grained interactions. Extensive experiments validate the
  superiority of Story-Adapter in improving both semantic consistency and
  generative capability for fine-grained interactions, particularly in long story
  scenarios. The project page and associated code can be accessed via
  https://jwmao1.github.io/storyadapter .
paper_ai_keywords: diffusion models semantic consistency text-to-image models global reference cross-attention module global embeddings iterative paradigm generative capability fine-grained interactions