title: Compass Control: Multi Object Orientation Control for Text-to-Image
    Generation
summary: Existing approaches for controlling text-to-image diffusion models, while
  powerful, do not allow for explicit 3D object-centric control, such as precise
  control of object orientation. In this work, we address the problem of
  multi-object orientation control in text-to-image diffusion models. This
  enables the generation of diverse multi-object scenes with precise orientation
  control for each object. The key idea is to condition the diffusion model with
  a set of orientation-aware compass tokens, one for each object, along
  with text tokens. A light-weight encoder network predicts these compass tokens
  taking object orientation as the input. The model is trained on a synthetic
  dataset of procedurally generated scenes, each containing one or two 3D assets
  on a plain background. However, direct training this framework results in poor
  orientation control as well as leads to entanglement among objects. To mitigate
  this, we intervene in the generation process and constrain the cross-attention
  maps of each compass token to its corresponding object regions. The trained
  model is able to achieve precise orientation control for a) complex objects not
  seen during training and b) multi-object scenes with more than two objects,
  indicating strong generalization capabilities. Further, when combined with
  personalization methods, our method precisely controls the orientation of the
  new object in diverse contexts. Our method achieves state-of-the-art
  orientation control and text alignment, quantified with extensive evaluations
  and a user study.
paper_authors._id: 67f8c9f028f4f60ceb4103e4
paper_authors.name: Rishubh Parihar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:43:15.754Z
paper_authors._id: 631af10b492c0c57a22ea3e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662710000523-noauth.png
paper_authors.fullname: Rishubh Parihar
paper_authors.type: user
paper_authors.user: RishubhPar
paper_authors._id: 67f8c9f028f4f60ceb4103e5
paper_authors.name: Vaibhav Agrawal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-17T08:05:25.600Z
paper_authors._id: 67ab8a5cc8e2f51341bc52f4
paper_authors.avatarUrl: /avatars/99ee27bb922d04d098da17db902b3700.svg
paper_authors.fullname: Vaibhav Agrawal
paper_authors.type: user
paper_authors.user: va1bhavagrawa1
paper_authors._id: 67f8c9f028f4f60ceb4103e6
paper_authors.name: Sachidanand VS
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:43:46.850Z
paper_authors._id: 64f56843ad81562b476ec05d
paper_authors.avatarUrl: /avatars/fa90a685259f89bd1b447ba93359187e.svg
paper_authors.fullname: sachidanand vs
paper_authors.type: user
paper_authors.user: sachi1
paper_authors._id: 67f8c9f028f4f60ceb4103e7
paper_authors.name: R. Venkatesh Babu
paper_title: Compass Control: Multi Object Orientation Control for Text-to-Image
    Generation
paper_summary: Existing approaches for controlling text-to-image diffusion models, while
  powerful, do not allow for explicit 3D object-centric control, such as precise
  control of object orientation. In this work, we address the problem of
  multi-object orientation control in text-to-image diffusion models. This
  enables the generation of diverse multi-object scenes with precise orientation
  control for each object. The key idea is to condition the diffusion model with
  a set of orientation-aware compass tokens, one for each object, along
  with text tokens. A light-weight encoder network predicts these compass tokens
  taking object orientation as the input. The model is trained on a synthetic
  dataset of procedurally generated scenes, each containing one or two 3D assets
  on a plain background. However, direct training this framework results in poor
  orientation control as well as leads to entanglement among objects. To mitigate
  this, we intervene in the generation process and constrain the cross-attention
  maps of each compass token to its corresponding object regions. The trained
  model is able to achieve precise orientation control for a) complex objects not
  seen during training and b) multi-object scenes with more than two objects,
  indicating strong generalization capabilities. Further, when combined with
  personalization methods, our method precisely controls the orientation of the
  new object in diverse contexts. Our method achieves state-of-the-art
  orientation control and text alignment, quantified with extensive evaluations
  and a user study.
paper_ai_summary: This work addresses precise 3D object orientation control in text-to-image diffusion models by introducing orientation-aware compass tokens and constraining cross-attention maps.
paper_ai_keywords: text-to-image diffusion models multi-object orientation control compass tokens light-weight encoder cross-attention maps orientation control procedural generation generalization capabilities personalization methods text alignment