title: IA-T2I: Internet-Augmented Text-to-Image Generation
summary: Current text-to-image (T2I) generation models achieve promising results, but
  they fail on the scenarios where the knowledge implied in the text prompt is
  uncertain. For example, a T2I model released in February would struggle to
  generate a suitable poster for a movie premiering in April, because the
  character designs and styles are uncertain to the model. To solve this problem,
  we propose an Internet-Augmented text-to-image generation (IA-T2I) framework to
  compel T2I models clear about such uncertain knowledge by providing them with
  reference images. Specifically, an active retrieval module is designed to
  determine whether a reference image is needed based on the given text prompt; a
  hierarchical image selection module is introduced to find the most suitable
  image returned by an image search engine to enhance the T2I model; a
  self-reflection mechanism is presented to continuously evaluate and refine the
  generated image to ensure faithful alignment with the text prompt. To evaluate
  the proposed framework's performance, we collect a dataset named Img-Ref-T2I,
  where text prompts include three types of uncertain knowledge: (1) known but
  rare. (2) unknown. (3) ambiguous. Moreover, we carefully craft a complex prompt
  to guide GPT-4o in making preference evaluation, which has been shown to have
  an evaluation accuracy similar to that of human preference evaluation.
  Experimental results demonstrate the effectiveness of our framework,
  outperforming GPT-4o by about 30% in human evaluation.
paper_authors._id: 682e9d84a0db46260ccc15e1
paper_authors.name: Chuanhao Li
paper_authors._id: 682e9d84a0db46260ccc15e2
paper_authors.name: Jianwen Sun
paper_authors._id: 682e9d84a0db46260ccc15e3
paper_authors.name: Yukang Feng
paper_authors._id: 682e9d84a0db46260ccc15e4
paper_authors.name: Mingliang Zhai
paper_authors._id: 682e9d84a0db46260ccc15e5
paper_authors.name: Yifan Chang
paper_authors._id: 682e9d84a0db46260ccc15e6
paper_authors.name: Kaipeng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:49:38.607Z
paper_authors._id: 65f1713552c38a91e0a445e8
paper_authors.avatarUrl: /avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
paper_authors.fullname: kaipeng
paper_authors.type: user
paper_authors.user: kpzhang996
paper_title: IA-T2I: Internet-Augmented Text-to-Image Generation
paper_summary: Current text-to-image (T2I) generation models achieve promising results, but
  they fail on the scenarios where the knowledge implied in the text prompt is
  uncertain. For example, a T2I model released in February would struggle to
  generate a suitable poster for a movie premiering in April, because the
  character designs and styles are uncertain to the model. To solve this problem,
  we propose an Internet-Augmented text-to-image generation (IA-T2I) framework to
  compel T2I models clear about such uncertain knowledge by providing them with
  reference images. Specifically, an active retrieval module is designed to
  determine whether a reference image is needed based on the given text prompt; a
  hierarchical image selection module is introduced to find the most suitable
  image returned by an image search engine to enhance the T2I model; a
  self-reflection mechanism is presented to continuously evaluate and refine the
  generated image to ensure faithful alignment with the text prompt. To evaluate
  the proposed framework's performance, we collect a dataset named Img-Ref-T2I,
  where text prompts include three types of uncertain knowledge: (1) known but
  rare. (2) unknown. (3) ambiguous. Moreover, we carefully craft a complex prompt
  to guide GPT-4o in making preference evaluation, which has been shown to have
  an evaluation accuracy similar to that of human preference evaluation.
  Experimental results demonstrate the effectiveness of our framework,
  outperforming GPT-4o by about 30% in human evaluation.
paper_ai_summary: An Internet-Augmented text-to-image generation framework improves uncertain text prompt handling by integrating reference images, enhancing image quality and fidelity.
paper_ai_keywords: text-to-image generation IA-T2I framework reference images active retrieval module hierarchical image selection module self-reflection mechanism Img-Ref-T2I dataset GPT-4o