title: MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual
    Storytelling via Multi-Layered Semantic-Aware Denoising
summary: Visual storytelling often uses nontypical aspect-ratio images like scroll
  paintings, comic strips, and panoramas to create an expressive and compelling
  narrative. While generative AI has achieved great success and shown the
  potential to reshape the creative industry, it remains a challenge to generate
  coherent and engaging content with arbitrary size and controllable style,
  concept, and layout, all of which are essential for visual storytelling. To
  overcome the shortcomings of previous methods including repetitive content,
  style inconsistency, and lack of controllability, we propose MagicScroll, a
  multi-layered, progressive diffusion-based image generation framework with a
  novel semantic-aware denoising process. The model enables fine-grained control
  over the generated image on object, scene, and background levels with text,
  image, and layout conditions. We also establish the first benchmark for
  nontypical aspect-ratio image generation for visual storytelling including
  mediums like paintings, comics, and cinematic panoramas, with customized
  metrics for systematic evaluation. Through comparative and ablation studies,
  MagicScroll showcases promising results in aligning with the narrative text,
  improving visual coherence, and engaging the audience. We plan to release the
  code and benchmark in the hope of a better collaboration between AI researchers
  and creative practitioners involving visual storytelling.
paper_authors._id: 65812ffd4cdbd00e3de43a22
paper_authors.name: Bingyuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T13:48:41.010Z
paper_authors._id: 631c01a19edb6d320a16fe8f
paper_authors.avatarUrl: /avatars/93f04a678d7d1f78decfb279767e42d0.svg
paper_authors.fullname: Bingyuan Wang
paper_authors.type: user
paper_authors.user: BingyuanWang
paper_authors._id: 65812ffd4cdbd00e3de43a23
paper_authors.name: Hengyu Meng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:37:37.088Z
paper_authors._id: 63edfe1c5f1300034dd90a37
paper_authors.avatarUrl: /avatars/fa742c6da029ce73b395d1a12a97753f.svg
paper_authors.fullname: Hyuno
paper_authors.type: user
paper_authors.user: HengyuMeng
paper_authors._id: 65812ffd4cdbd00e3de43a24
paper_authors.name: Zeyu Cai
paper_authors._id: 65812ffd4cdbd00e3de43a25
paper_authors.name: Lanjiong Li
paper_authors._id: 65812ffd4cdbd00e3de43a26
paper_authors.name: Yue Ma
paper_authors._id: 65812ffd4cdbd00e3de43a27
paper_authors.name: Qifeng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T09:38:26.050Z
paper_authors._id: 6467b121e7a6a374fd19b44b
paper_authors.avatarUrl: /avatars/3f2874d58986d651aef55e3408b05700.svg
paper_authors.fullname: Qifeng Chen
paper_authors.type: user
paper_authors.user: cqf
paper_authors._id: 65812ffd4cdbd00e3de43a28
paper_authors.name: Zeyu Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-19T16:01:25.896Z
paper_authors._id: 6581a37c7d2dbe3781d48ac9
paper_authors.avatarUrl: /avatars/95211723e67434a29082bf6c29bae8e3.svg
paper_authors.fullname: Zeyu Wang
paper_authors.type: user
paper_authors.user: zachzeyuwang
paper_title: MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual
    Storytelling via Multi-Layered Semantic-Aware Denoising
paper_summary: Visual storytelling often uses nontypical aspect-ratio images like scroll
  paintings, comic strips, and panoramas to create an expressive and compelling
  narrative. While generative AI has achieved great success and shown the
  potential to reshape the creative industry, it remains a challenge to generate
  coherent and engaging content with arbitrary size and controllable style,
  concept, and layout, all of which are essential for visual storytelling. To
  overcome the shortcomings of previous methods including repetitive content,
  style inconsistency, and lack of controllability, we propose MagicScroll, a
  multi-layered, progressive diffusion-based image generation framework with a
  novel semantic-aware denoising process. The model enables fine-grained control
  over the generated image on object, scene, and background levels with text,
  image, and layout conditions. We also establish the first benchmark for
  nontypical aspect-ratio image generation for visual storytelling including
  mediums like paintings, comics, and cinematic panoramas, with customized
  metrics for systematic evaluation. Through comparative and ablation studies,
  MagicScroll showcases promising results in aligning with the narrative text,
  improving visual coherence, and engaging the audience. We plan to release the
  code and benchmark in the hope of a better collaboration between AI researchers
  and creative practitioners involving visual storytelling.
paper_ai_keywords: diffusion-based image generation semantic-aware denoising process