Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image
  Synthesis
Addressing the limitations of text as a source of accurate layout
representation in text-conditional diffusion models, many works incorporate
additional signals to condition certain attributes within a generated image.
Although successful, previous works do not account for the specific
localization of said attributes extended into the three dimensional plane. In
this context, we present a conditional diffusion model that integrates control
over three-dimensional object placement with disentangled representations of
global stylistic semantics from multiple exemplar images. Specifically, we
first introduce depth disentanglement training to leverage the
relative depth of objects as an estimator, allowing the model to identify the
absolute positions of unseen objects through the use of synthetic image
triplets. We also introduce soft guidance, a method for imposing
global semantics onto targeted regions without the use of any additional
localization cues. Our integrated framework, Compose and Conquer
(CnC), unifies these techniques to localize multiple conditions in a
disentangled manner. We demonstrate that our approach allows perception of
objects at varying depths while offering a versatile framework for composing
localized objects with different global semantics. Code:
https://github.com/tomtom1103/compose-and-conquer/
65a8bab30dba6b99a0fd765a
Jonghyun Lee
admin_assigned
2024-02-02T08:00:28.189Z
63834f5262badff4326ad517
/avatars/a747bcf121abb55f6d403cb1b1fae537.svg
Jonghyun Lee
user
tomtom1103
65a8bab30dba6b99a0fd765b
Hansam Cho
admin_assigned
2024-01-18T10:18:03.259Z
6355c80a0d6e89270d495994
/avatars/d25b0d5b47cb92cc6b498ef4e92f3f70.svg
Hansam Cho
user
Hansam
65a8bab30dba6b99a0fd765c
Youngjoon Yoo
admin_assigned
2024-01-18T10:19:05.738Z
6582d1b85def15befa412a88
/avatars/5104073d752a9ed1a57678dd70753790.svg
YoungJoon Yoo
user
yjyoo3312
65a8bab30dba6b99a0fd765d
Seoung Bum Kim
65a8bab30dba6b99a0fd765e
Yonghyun Jeong
Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image
  Synthesis
Addressing the limitations of text as a source of accurate layout
representation in text-conditional diffusion models, many works incorporate
additional signals to condition certain attributes within a generated image.
Although successful, previous works do not account for the specific
localization of said attributes extended into the three dimensional plane. In
this context, we present a conditional diffusion model that integrates control
over three-dimensional object placement with disentangled representations of
global stylistic semantics from multiple exemplar images. Specifically, we
first introduce depth disentanglement training to leverage the
relative depth of objects as an estimator, allowing the model to identify the
absolute positions of unseen objects through the use of synthetic image
triplets. We also introduce soft guidance, a method for imposing
global semantics onto targeted regions without the use of any additional
localization cues. Our integrated framework, Compose and Conquer
(CnC), unifies these techniques to localize multiple conditions in a
disentangled manner. We demonstrate that our approach allows perception of
objects at varying depths while offering a versatile framework for composing
localized objects with different global semantics. Code:
https://github.com/tomtom1103/compose-and-conquer/
A conditional diffusion model with depth disentanglement and soft guidance improves localization and semantic composition of objects in generated images.
diffusion models depth disentanglement soft guidance relative depth synthetic image triplets global stylistic semantics disentangled representations object localization