title: Make It Count: Text-to-Image Generation with an Accurate Number of
    Objects
summary: Despite the unprecedented success of text-to-image diffusion models,
  controlling the number of depicted objects using text is surprisingly hard.
  This is important for various applications from technical documents, to
  children's books to illustrating cooking recipes. Generating object-correct
  counts is fundamentally challenging because the generative model needs to keep
  a sense of separate identity for every instance of the object, even if several
  objects look identical or overlap, and then carry out a global computation
  implicitly during generation. It is still unknown if such representations
  exist. To address count-correct generation, we first identify features within
  the diffusion model that can carry the object identity information. We then use
  them to separate and count instances of objects during the denoising process
  and detect over-generation and under-generation. We fix the latter by training
  a model that predicts both the shape and location of a missing object, based on
  the layout of existing ones, and show how it can be used to guide denoising
  with correct object count. Our approach, CountGen, does not depend on external
  source to determine object layout, but rather uses the prior from the diffusion
  model itself, creating prompt-dependent and seed-dependent layouts. Evaluated
  on two benchmark datasets, we find that CountGen strongly outperforms the
  count-accuracy of existing baselines.
paper_authors._id: 666fe1d5b955b0e6551b19a8
paper_authors.name: Lital Binyamin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-17T14:11:52.674Z
paper_authors._id: 66701c543f16ff3d4540681e
paper_authors.avatarUrl: /avatars/cfdfd398d4600dad3a17c20424033a6b.svg
paper_authors.fullname: Lital Binyamin
paper_authors.type: user
paper_authors.user: Litalby
paper_authors._id: 666fe1d5b955b0e6551b19a9
paper_authors.name: Yoad Tewel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-17T14:13:05.460Z
paper_authors._id: 6201838f305742efd5a6a0e3
paper_authors.avatarUrl: /avatars/4e47ab9ef1a4b9f59756a6d8c90a970f.svg
paper_authors.fullname: Yoad Tewel
paper_authors.type: user
paper_authors.user: YoadTew
paper_authors._id: 666fe1d5b955b0e6551b19aa
paper_authors.name: Hilit Segev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-17T14:13:11.770Z
paper_authors._id: 6492bc477ddf9471f55de3b8
paper_authors.avatarUrl: /avatars/ba42cf14b1ae558bd29e3ff51e59531a.svg
paper_authors.fullname: Hilit Segev
paper_authors.type: user
paper_authors.user: Hilit
paper_authors._id: 666fe1d5b955b0e6551b19ab
paper_authors.name: Eran Hirsch
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T10:45:26.366Z
paper_authors._id: 609eb1fc1172dedeac2200db
paper_authors.avatarUrl: /avatars/a7384a8d3389610b38388c100a28c86d.svg
paper_authors.fullname: H
paper_authors.type: user
paper_authors.user: Eran
paper_authors._id: 666fe1d5b955b0e6551b19ac
paper_authors.name: Royi Rassin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-17T14:13:21.522Z
paper_authors._id: 62a7581cf049be35252a2e7c
paper_authors.avatarUrl: /avatars/91de4eb48f51bfd6e028c08ccfa98f8c.svg
paper_authors.fullname: Royi Rassin
paper_authors.type: user
paper_authors.user: Royir
paper_authors._id: 666fe1d5b955b0e6551b19ad
paper_authors.name: Gal Chechik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-17T14:13:27.138Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_title: Make It Count: Text-to-Image Generation with an Accurate Number of
    Objects
paper_summary: Despite the unprecedented success of text-to-image diffusion models,
  controlling the number of depicted objects using text is surprisingly hard.
  This is important for various applications from technical documents, to
  children's books to illustrating cooking recipes. Generating object-correct
  counts is fundamentally challenging because the generative model needs to keep
  a sense of separate identity for every instance of the object, even if several
  objects look identical or overlap, and then carry out a global computation
  implicitly during generation. It is still unknown if such representations
  exist. To address count-correct generation, we first identify features within
  the diffusion model that can carry the object identity information. We then use
  them to separate and count instances of objects during the denoising process
  and detect over-generation and under-generation. We fix the latter by training
  a model that predicts both the shape and location of a missing object, based on
  the layout of existing ones, and show how it can be used to guide denoising
  with correct object count. Our approach, CountGen, does not depend on external
  source to determine object layout, but rather uses the prior from the diffusion
  model itself, creating prompt-dependent and seed-dependent layouts. Evaluated
  on two benchmark datasets, we find that CountGen strongly outperforms the
  count-accuracy of existing baselines.
paper_ai_summary: CountGen enhances text-to-image diffusion models by accurately controlling the number of depicted objects using features that carry object identity information, surpassing existing methods on benchmark datasets.
paper_ai_keywords: diffusion models denoising process object identity information count-correct generation CountGen