title: BeyondScene: Higher-Resolution Human-Centric Scene Generation With
    Pretrained Diffusion
summary: Generating higher-resolution human-centric scenes with details and controls
  remains a challenge for existing text-to-image diffusion models. This challenge
  stems from limited training image size, text encoder capacity (limited tokens),
  and the inherent difficulty of generating complex scenes involving multiple
  humans. While current methods attempted to address training size limit only,
  they often yielded human-centric scenes with severe artifacts. We propose
  BeyondScene, a novel framework that overcomes prior limitations, generating
  exquisite higher-resolution (over 8K) human-centric scenes with exceptional
  text-image correspondence and naturalness using existing pretrained diffusion
  models. BeyondScene employs a staged and hierarchical approach to initially
  generate a detailed base image focusing on crucial elements in instance
  creation for multiple humans and detailed descriptions beyond token limit of
  diffusion model, and then to seamlessly convert the base image to a
  higher-resolution output, exceeding training image size and incorporating
  details aware of text and instances via our novel instance-aware hierarchical
  enlargement process that consists of our proposed high-frequency injected
  forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing
  methods in terms of correspondence with detailed text descriptions and
  naturalness, paving the way for advanced applications in higher-resolution
  human-centric scene creation beyond the capacity of pretrained diffusion models
  without costly retraining. Project page:
  https://janeyeon.github.io/beyond-scene.
paper_authors._id: 6614c5a1cc60e737481bfa06
paper_authors.name: Gwanghyun Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-09T19:48:34.302Z
paper_authors._id: 632c261b16faa31b24d82847
paper_authors.avatarUrl: /avatars/5a92de91137d74a65ab5fda89135d713.svg
paper_authors.fullname: Gwanghyun (Bradley) Kim
paper_authors.type: user
paper_authors.user: gwang-kim
paper_authors._id: 6614c5a1cc60e737481bfa07
paper_authors.name: Hayeon Kim
paper_authors._id: 6614c5a1cc60e737481bfa08
paper_authors.name: Hoigi Seo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-09T19:48:27.249Z
paper_authors._id: 633e6f07309a99325095dd42
paper_authors.avatarUrl: /avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
paper_authors.fullname: Hoigi Seo
paper_authors.type: user
paper_authors.user: Agorium
paper_authors._id: 6614c5a1cc60e737481bfa09
paper_authors.name: Dong Un Kang
paper_authors._id: 6614c5a1cc60e737481bfa0a
paper_authors.name: Se Young Chun
paper_title: BeyondScene: Higher-Resolution Human-Centric Scene Generation With
    Pretrained Diffusion
paper_summary: Generating higher-resolution human-centric scenes with details and controls
  remains a challenge for existing text-to-image diffusion models. This challenge
  stems from limited training image size, text encoder capacity (limited tokens),
  and the inherent difficulty of generating complex scenes involving multiple
  humans. While current methods attempted to address training size limit only,
  they often yielded human-centric scenes with severe artifacts. We propose
  BeyondScene, a novel framework that overcomes prior limitations, generating
  exquisite higher-resolution (over 8K) human-centric scenes with exceptional
  text-image correspondence and naturalness using existing pretrained diffusion
  models. BeyondScene employs a staged and hierarchical approach to initially
  generate a detailed base image focusing on crucial elements in instance
  creation for multiple humans and detailed descriptions beyond token limit of
  diffusion model, and then to seamlessly convert the base image to a
  higher-resolution output, exceeding training image size and incorporating
  details aware of text and instances via our novel instance-aware hierarchical
  enlargement process that consists of our proposed high-frequency injected
  forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing
  methods in terms of correspondence with detailed text descriptions and
  naturalness, paving the way for advanced applications in higher-resolution
  human-centric scene creation beyond the capacity of pretrained diffusion models
  without costly retraining. Project page:
  https://janeyeon.github.io/beyond-scene.
paper_ai_keywords: BeyondScene diffusion models text-to-image diffusion models hierarchical approach high-frequency injected forward diffusion adaptive joint diffusion instance-aware hierarchical enlargement process higher-resolution output text-image correspondence higher-resolution human-centric scenes instance-aware