title: Do I look like a `cat.n.01` to you? A Taxonomy Image Generation
    Benchmark
summary: This paper explores the feasibility of using text-to-image models in a
  zero-shot setup to generate images for taxonomy concepts. While text-based
  methods for taxonomy enrichment are well-established, the potential of the
  visual dimension remains unexplored. To address this, we propose a
  comprehensive benchmark for Taxonomy Image Generation that assesses models'
  abilities to understand taxonomy concepts and generate relevant, high-quality
  images. The benchmark includes common-sense and randomly sampled WordNet
  concepts, alongside the LLM generated predictions. The 12 models are evaluated
  using 9 novel taxonomy-related text-to-image metrics and human feedback.
  Moreover, we pioneer the use of pairwise evaluation with GPT-4 feedback for
  image generation. Experimental results show that the ranking of models differs
  significantly from standard T2I tasks. Playground-v2 and FLUX consistently
  outperform across metrics and subsets and the retrieval-based approach performs
  poorly. These findings highlight the potential for automating the curation of
  structured data resources.
paper_authors._id: 67d3da3ce0b767b3d0fae2a4
paper_authors.name: Viktor Moskvoretskii
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-14T07:38:54.624Z
paper_authors._id: 63bbfd74141c7d395c471768
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
paper_authors.fullname: Viktor Moskvoretskii
paper_authors.type: user
paper_authors.user: VityaVitalich
paper_authors._id: 67d3da3ce0b767b3d0fae2a5
paper_authors.name: Alina Lobanova
paper_authors._id: 67d3da3ce0b767b3d0fae2a6
paper_authors.name: Ekaterina Neminova
paper_authors._id: 67d3da3ce0b767b3d0fae2a7
paper_authors.name: Chris Biemann
paper_authors._id: 67d3da3ce0b767b3d0fae2a8
paper_authors.name: Alexander Panchenko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T10:57:38.630Z
paper_authors._id: 605473729d7c1d4d81b7e52b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662046050710-605473729d7c1d4d81b7e52b.jpeg
paper_authors.fullname: Alexander Panchenko
paper_authors.type: user
paper_authors.user: apanc
paper_authors._id: 67d3da3ce0b767b3d0fae2a9
paper_authors.name: Irina Nikishina
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T10:57:44.989Z
paper_authors._id: 6458a9ed6fa580137af4c710
paper_authors.avatarUrl: /avatars/8b9f99537adf30fee6aa7ae61ab7235b.svg
paper_authors.fullname: Irina Nikishina
paper_authors.type: user
paper_authors.user: lilaspourpre
paper_title: Do I look like a `cat.n.01` to you? A Taxonomy Image Generation
    Benchmark
paper_summary: This paper explores the feasibility of using text-to-image models in a
  zero-shot setup to generate images for taxonomy concepts. While text-based
  methods for taxonomy enrichment are well-established, the potential of the
  visual dimension remains unexplored. To address this, we propose a
  comprehensive benchmark for Taxonomy Image Generation that assesses models'
  abilities to understand taxonomy concepts and generate relevant, high-quality
  images. The benchmark includes common-sense and randomly sampled WordNet
  concepts, alongside the LLM generated predictions. The 12 models are evaluated
  using 9 novel taxonomy-related text-to-image metrics and human feedback.
  Moreover, we pioneer the use of pairwise evaluation with GPT-4 feedback for
  image generation. Experimental results show that the ranking of models differs
  significantly from standard T2I tasks. Playground-v2 and FLUX consistently
  outperform across metrics and subsets and the retrieval-based approach performs
  poorly. These findings highlight the potential for automating the curation of
  structured data resources.
paper_ai_keywords: text-to-image models zero-shot setup taxonomy concepts text-based methods taxonomy enrichment comprehensive benchmark WordNet LLM generated predictions taxonomy-related text-to-image metrics pairwise evaluation GPT-4 feedback image generation ranking of models retrieval-based approach automating the curation of structured data resources