title: WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image
    Generation
summary: Text-to-Image (T2I) models are capable of generating high-quality artistic
  creations and visual content. However, existing research and evaluation
  standards predominantly focus on image realism and shallow text-image
  alignment, lacking a comprehensive assessment of complex semantic understanding
  and world knowledge integration in text to image generation. To address this
  challenge, we propose WISE, the first benchmark specifically
  designed for World Knowledge-Informed Semantic
  Evaluation. WISE moves beyond simple word-pixel mapping by
  challenging models with 1000 meticulously crafted prompts across 25 sub-domains
  in cultural common sense, spatio-temporal reasoning, and natural science. To
  overcome the limitations of traditional CLIP metric, we introduce
  WiScore, a novel quantitative metric for assessing knowledge-image
  alignment. Through comprehensive testing of 20 models (10 dedicated T2I models
  and 10 unified multimodal models) using 1,000 structured prompts spanning 25
  subdomains, our findings reveal significant limitations in their ability to
  effectively integrate and apply world knowledge during image generation,
  highlighting critical pathways for enhancing knowledge incorporation and
  application in next-generation T2I models. Code and data are available at
  https://github.com/PKU-YuanGroup/WISE.
paper_authors._id: 67d01cf289beb7795a51f111
paper_authors.name: Yuwei Niu
paper_authors._id: 67d01cf289beb7795a51f112
paper_authors.name: Munan Ning
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:00.839Z
paper_authors._id: 65e14c28b1a6de8a71e70172
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65e14c28b1a6de8a71e70172/D097SILGsqoufpp3sG8tV.jpeg
paper_authors.fullname: Munan Ning
paper_authors.type: user
paper_authors.user: MunanNing
paper_authors._id: 67d01cf289beb7795a51f113
paper_authors.name: Mengren Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:12.978Z
paper_authors._id: 66c16d5d780d735f17bccfdc
paper_authors.avatarUrl: /avatars/6bff393b61a78b425d6f429a6d75ad56.svg
paper_authors.fullname: Mengren Zheng
paper_authors.type: user
paper_authors.user: zmr66z6xx6
paper_authors._id: 67d01cf289beb7795a51f114
paper_authors.name: Bin Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:05:29.423Z
paper_authors._id: 6367a8175bb06007ea099b8f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6367a8175bb06007ea099b8f/IjG7HyWyWRlVt_XwRbxRW.jpeg
paper_authors.fullname: linbin
paper_authors.type: user
paper_authors.user: LanguageBind
paper_authors._id: 67d01cf289beb7795a51f115
paper_authors.name: Peng Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:20.825Z
paper_authors._id: 63ad0b04e3b217fb36d36c13
paper_authors.avatarUrl: /avatars/5a3715ba20859052ba04c048db9e03c2.svg
paper_authors.fullname: Peng Jin
paper_authors.type: user
paper_authors.user: Pengjin
paper_authors._id: 67d01cf289beb7795a51f116
paper_authors.name: Jiaqi Liao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:32.257Z
paper_authors._id: 6630b287ce65a66ed8f04eba
paper_authors.avatarUrl: /avatars/18ad0e3b453021ea6006e23333e92c12.svg
paper_authors.fullname: liaojiaqi
paper_authors.type: user
paper_authors.user: ljq940913
paper_authors._id: 67d01cf289beb7795a51f117
paper_authors.name: Kunpeng Ning
paper_authors._id: 67d01cf289beb7795a51f118
paper_authors.name: Bin Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:47.289Z
paper_authors._id: 6476e69d99a5ce743ccde294
paper_authors.avatarUrl: /avatars/12955bcec780ff34a7e78281b45b016d.svg
paper_authors.fullname: Bin Zhu
paper_authors.type: user
paper_authors.user: binzhu2023
paper_authors._id: 67d01cf289beb7795a51f119
paper_authors.name: Li Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:56:54.455Z
paper_authors._id: 614030bd8cbdb613b82f36a8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631596713749-noauth.jpeg
paper_authors.fullname: Li Yuan
paper_authors.type: user
paper_authors.user: LiYuan
paper_title: WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image
    Generation
paper_summary: Text-to-Image (T2I) models are capable of generating high-quality artistic
  creations and visual content. However, existing research and evaluation
  standards predominantly focus on image realism and shallow text-image
  alignment, lacking a comprehensive assessment of complex semantic understanding
  and world knowledge integration in text to image generation. To address this
  challenge, we propose WISE, the first benchmark specifically
  designed for World Knowledge-Informed Semantic
  Evaluation. WISE moves beyond simple word-pixel mapping by
  challenging models with 1000 meticulously crafted prompts across 25 sub-domains
  in cultural common sense, spatio-temporal reasoning, and natural science. To
  overcome the limitations of traditional CLIP metric, we introduce
  WiScore, a novel quantitative metric for assessing knowledge-image
  alignment. Through comprehensive testing of 20 models (10 dedicated T2I models
  and 10 unified multimodal models) using 1,000 structured prompts spanning 25
  subdomains, our findings reveal significant limitations in their ability to
  effectively integrate and apply world knowledge during image generation,
  highlighting critical pathways for enhancing knowledge incorporation and
  application in next-generation T2I models. Code and data are available at
  https://github.com/PKU-YuanGroup/WISE.
paper_ai_summary: WISE is a benchmark for evaluating text-to-image models' integration of world knowledge and semantic understanding through 1,000 prompts across various domains, challenging traditional metrics with WiScore.
paper_ai_keywords: Text-to-Image (T2I) semantic understanding world knowledge CLIP metric WiScore multimodal models image generation cultural common sense spatio-temporal reasoning natural science