BeyondScene: Higher-Resolution Human-Centric Scene Generation With
  Pretrained Diffusion
Generating higher-resolution human-centric scenes with details and controls
remains a challenge for existing text-to-image diffusion models. This challenge
stems from limited training image size, text encoder capacity (limited tokens),
and the inherent difficulty of generating complex scenes involving multiple
humans. While current methods attempted to address training size limit only,
they often yielded human-centric scenes with severe artifacts. We propose
BeyondScene, a novel framework that overcomes prior limitations, generating
exquisite higher-resolution (over 8K) human-centric scenes with exceptional
text-image correspondence and naturalness using existing pretrained diffusion
models. BeyondScene employs a staged and hierarchical approach to initially
generate a detailed base image focusing on crucial elements in instance
creation for multiple humans and detailed descriptions beyond token limit of
diffusion model, and then to seamlessly convert the base image to a
higher-resolution output, exceeding training image size and incorporating
details aware of text and instances via our novel instance-aware hierarchical
enlargement process that consists of our proposed high-frequency injected
forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing
methods in terms of correspondence with detailed text descriptions and
naturalness, paving the way for advanced applications in higher-resolution
human-centric scene creation beyond the capacity of pretrained diffusion models
without costly retraining. Project page:
https://janeyeon.github.io/beyond-scene.
6614c5a1cc60e737481bfa06
Gwanghyun Kim
claimed_verified
2024-04-09T19:48:34.302Z
632c261b16faa31b24d82847
/avatars/5a92de91137d74a65ab5fda89135d713.svg
Gwanghyun (Bradley) Kim
user
gwang-kim
6614c5a1cc60e737481bfa07
Hayeon Kim
claimed_verified
2025-07-22T08:01:23.217Z
63b3fea57af2e415f258a0ea
/avatars/31f3e6871b8db4fe4d3d2859d9b4ac0a.svg
HayeonKim
user
yeonE
6614c5a1cc60e737481bfa08
Hoigi Seo
claimed_verified
2024-04-09T19:48:27.249Z
633e6f07309a99325095dd42
/avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
Hoigi Seo
user
Agorium
6614c5a1cc60e737481bfa09
Dong Un Kang
6614c5a1cc60e737481bfa0a
Se Young Chun
BeyondScene: Higher-Resolution Human-Centric Scene Generation With
  Pretrained Diffusion
Generating higher-resolution human-centric scenes with details and controls
remains a challenge for existing text-to-image diffusion models. This challenge
stems from limited training image size, text encoder capacity (limited tokens),
and the inherent difficulty of generating complex scenes involving multiple
humans. While current methods attempted to address training size limit only,
they often yielded human-centric scenes with severe artifacts. We propose
BeyondScene, a novel framework that overcomes prior limitations, generating
exquisite higher-resolution (over 8K) human-centric scenes with exceptional
text-image correspondence and naturalness using existing pretrained diffusion
models. BeyondScene employs a staged and hierarchical approach to initially
generate a detailed base image focusing on crucial elements in instance
creation for multiple humans and detailed descriptions beyond token limit of
diffusion model, and then to seamlessly convert the base image to a
higher-resolution output, exceeding training image size and incorporating
details aware of text and instances via our novel instance-aware hierarchical
enlargement process that consists of our proposed high-frequency injected
forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing
methods in terms of correspondence with detailed text descriptions and
naturalness, paving the way for advanced applications in higher-resolution
human-centric scene creation beyond the capacity of pretrained diffusion models
without costly retraining. Project page:
https://janeyeon.github.io/beyond-scene.
BeyondScene generates high-resolution human-centric scenes with detailed text descriptions and naturalness using pretrained diffusion models through a hierarchical enlargement process.
text-to-image diffusion models training image size text encoder pretrained diffusion models staged approach hierarchical approach instance-aware hierarchical enlargement process high-frequency injected forward diffusion adaptive joint diffusion