title: InstanceGen: Image Generation with Instance-level Instructions
summary: Despite rapid advancements in the capabilities of generative models,
  pretrained text-to-image models still struggle in capturing the semantics
  conveyed by complex prompts that compound multiple objects and instance-level
  attributes. Consequently, we are witnessing growing interests in integrating
  additional structural constraints, typically in the form of coarse bounding
  boxes, to better guide the generation process in such challenging cases. In
  this work, we take the idea of structural guidance a step further by making the
  observation that contemporary image generation models can directly provide a
  plausible fine-grained structural initialization. We propose a technique that
  couples this image-based structural guidance with LLM-based instance-level
  instructions, yielding output images that adhere to all parts of the text
  prompt, including object counts, instance-level attributes, and spatial
  relations between instances.
paper_authors._id: 6828f2e4159fce854b16f5fa
paper_authors.name: Etai Sella
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-17T20:34:50.662Z
paper_authors._id: 63b3e4b4b7fec0adf64ded69
paper_authors.avatarUrl: /avatars/55804581c263c1d68b6fc59a1b0a3df7.svg
paper_authors.fullname: Etai  Sella
paper_authors.type: user
paper_authors.user: etaisella
paper_authors._id: 6828f2e4159fce854b16f5fb
paper_authors.name: Yanir Kleiman
paper_authors._id: 6828f2e4159fce854b16f5fc
paper_authors.name: Hadar Averbuch-Elor
paper_title: InstanceGen: Image Generation with Instance-level Instructions
paper_summary: Despite rapid advancements in the capabilities of generative models,
  pretrained text-to-image models still struggle in capturing the semantics
  conveyed by complex prompts that compound multiple objects and instance-level
  attributes. Consequently, we are witnessing growing interests in integrating
  additional structural constraints, typically in the form of coarse bounding
  boxes, to better guide the generation process in such challenging cases. In
  this work, we take the idea of structural guidance a step further by making the
  observation that contemporary image generation models can directly provide a
  plausible fine-grained structural initialization. We propose a technique that
  couples this image-based structural guidance with LLM-based instance-level
  instructions, yielding output images that adhere to all parts of the text
  prompt, including object counts, instance-level attributes, and spatial
  relations between instances.
paper_ai_summary: The proposed technique combines image-based structural guidance with LLM-based instructions to align generated images with complex, detailed text prompts.
paper_ai_keywords: pretrained text-to-image models semantic prompts coarse bounding boxes fine-grained structural initialization instance-level instructions object counts instance-level attributes spatial relations