title: IMAGINE-E: Image Generation Intelligence Evaluation of State-of-the-art
    Text-to-Image Models
summary: With the rapid development of diffusion models, text-to-image(T2I) models
  have made significant progress, showcasing impressive abilities in prompt
  following and image generation. Recently launched models such as FLUX.1 and
  Ideogram2.0, along with others like Dall-E3 and Stable Diffusion 3, have
  demonstrated exceptional performance across various complex tasks, raising
  questions about whether T2I models are moving towards general-purpose
  applicability. Beyond traditional image generation, these models exhibit
  capabilities across a range of fields, including controllable generation, image
  editing, video, audio, 3D, and motion generation, as well as computer vision
  tasks like semantic segmentation and depth estimation. However, current
  evaluation frameworks are insufficient to comprehensively assess these models'
  performance across expanding domains. To thoroughly evaluate these models, we
  developed the IMAGINE-E and tested six prominent models: FLUX.1, Ideogram2.0,
  Midjourney, Dall-E3, Stable Diffusion 3, and Jimeng. Our evaluation is divided
  into five key domains: structured output generation, realism, and physical
  consistency, specific domain generation, challenging scenario generation, and
  multi-style creation tasks. This comprehensive assessment highlights each
  model's strengths and limitations, particularly the outstanding performance of
  FLUX.1 and Ideogram2.0 in structured and specific domain tasks, underscoring
  the expanding applications and potential of T2I models as foundational AI
  tools. This study provides valuable insights into the current state and future
  trajectory of T2I models as they evolve towards general-purpose usability.
  Evaluation scripts will be released at https://github.com/jylei16/Imagine-e.
paper_authors._id: 679316ff3698fd97252a8e6f
paper_authors.name: Jiayi Lei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:12:22.641Z
paper_authors._id: 64c3c72e8f31d1e6c664b052
paper_authors.avatarUrl: /avatars/af1ad5048eaa9dc417837ad02f927911.svg
paper_authors.fullname: jiayi lei
paper_authors.type: user
paper_authors.user: jyjyjyjy
paper_authors._id: 679316ff3698fd97252a8e70
paper_authors.name: Renrui Zhang
paper_authors._id: 679316ff3698fd97252a8e71
paper_authors.name: Xiangfei Hu
paper_authors._id: 679316ff3698fd97252a8e72
paper_authors.name: Weifeng Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:13:07.303Z
paper_authors._id: 66026c9068d519ed32519e9c
paper_authors.avatarUrl: /avatars/8fa051312c713772e5b8ba65989ff7f5.svg
paper_authors.fullname: Weifeng Lin
paper_authors.type: user
paper_authors.user: Afeng-x
paper_authors._id: 679316ff3698fd97252a8e73
paper_authors.name: Zhen Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T20:51:38.124Z
paper_authors._id: 6285a9133ab6642179158944
paper_authors.avatarUrl: /avatars/6e10fa07c94141fcdbe0cab02bb731ca.svg
paper_authors.fullname: Zhen Li
paper_authors.type: user
paper_authors.user: Paper99
paper_authors._id: 679316ff3698fd97252a8e74
paper_authors.name: Wenjian Sun
paper_authors._id: 679316ff3698fd97252a8e75
paper_authors.name: Ruoyi Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:13:21.861Z
paper_authors._id: 64a54586c0f13de8e7093314
paper_authors.avatarUrl: /avatars/389e43e9a32cf2fc95f8f3a23b8f0508.svg
paper_authors.fullname: Ruoyi Du
paper_authors.type: user
paper_authors.user: RuoyiDu
paper_authors._id: 679316ff3698fd97252a8e76
paper_authors.name: Le Zhuo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:13:27.523Z
paper_authors._id: 6358a167f56b03ec9147074d
paper_authors.avatarUrl: /avatars/e54ea7bf0c240cf76d538296efb3976c.svg
paper_authors.fullname: Le Zhuo
paper_authors.type: user
paper_authors.user: JackyZhuo
paper_authors._id: 679316ff3698fd97252a8e77
paper_authors.name: Zhongyu Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:13:33.108Z
paper_authors._id: 6740a5730bb4a675446a80ad
paper_authors.avatarUrl: /avatars/27c08e33df88e4f73c136da65f2b5adb.svg
paper_authors.fullname: Zhong-Yu Li
paper_authors.type: user
paper_authors.user: lzyhha
paper_authors._id: 679316ff3698fd97252a8e78
paper_authors.name: Xinyue Li
paper_authors._id: 679316ff3698fd97252a8e79
paper_authors.name: Shitian Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T13:30:14.688Z
paper_authors._id: 62c66504031996c36c86976a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62c66504031996c36c86976a/wIq0YJhkWnEhlzsh-TGYO.png
paper_authors.fullname: steve z
paper_authors.type: user
paper_authors.user: stzhao
paper_authors._id: 679316ff3698fd97252a8e7a
paper_authors.name: Ziyu Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:14:21.821Z
paper_authors._id: 647d9ab61a1fcad2fdbf2d3d
paper_authors.avatarUrl: /avatars/48c8aeae8979d2c87df8bde922437d62.svg
paper_authors.fullname: Ziyu Guo
paper_authors.type: user
paper_authors.user: ZiyuG
paper_authors._id: 679316ff3698fd97252a8e7b
paper_authors.name: Yiting Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:14:50.714Z
paper_authors._id: 6614fb3d5aed02b298a4b469
paper_authors.avatarUrl: /avatars/d0ddb4f989ad1a3f24128cc843347bde.svg
paper_authors.fullname: yiting lu
paper_authors.type: user
paper_authors.user: yeeeeeyy
paper_authors._id: 679316ff3698fd97252a8e7c
paper_authors.name: Peng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:15:04.489Z
paper_authors._id: 6759af3eccbc8817f9169179
paper_authors.avatarUrl: /avatars/49e64c7ccf71b8f25c52783b6ae93620.svg
paper_authors.fullname: Peng Gao
paper_authors.type: user
paper_authors.user: gaopenghigh
paper_authors._id: 679316ff3698fd97252a8e7d
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:15:11.437Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: IMAGINE-E: Image Generation Intelligence Evaluation of State-of-the-art
    Text-to-Image Models
paper_summary: With the rapid development of diffusion models, text-to-image(T2I) models
  have made significant progress, showcasing impressive abilities in prompt
  following and image generation. Recently launched models such as FLUX.1 and
  Ideogram2.0, along with others like Dall-E3 and Stable Diffusion 3, have
  demonstrated exceptional performance across various complex tasks, raising
  questions about whether T2I models are moving towards general-purpose
  applicability. Beyond traditional image generation, these models exhibit
  capabilities across a range of fields, including controllable generation, image
  editing, video, audio, 3D, and motion generation, as well as computer vision
  tasks like semantic segmentation and depth estimation. However, current
  evaluation frameworks are insufficient to comprehensively assess these models'
  performance across expanding domains. To thoroughly evaluate these models, we
  developed the IMAGINE-E and tested six prominent models: FLUX.1, Ideogram2.0,
  Midjourney, Dall-E3, Stable Diffusion 3, and Jimeng. Our evaluation is divided
  into five key domains: structured output generation, realism, and physical
  consistency, specific domain generation, challenging scenario generation, and
  multi-style creation tasks. This comprehensive assessment highlights each
  model's strengths and limitations, particularly the outstanding performance of
  FLUX.1 and Ideogram2.0 in structured and specific domain tasks, underscoring
  the expanding applications and potential of T2I models as foundational AI
  tools. This study provides valuable insights into the current state and future
  trajectory of T2I models as they evolve towards general-purpose usability.
  Evaluation scripts will be released at https://github.com/jylei16/Imagine-e.
paper_ai_keywords: diffusion models text-to-image (T2I) models prompt following image generation sequence-to-sequence controllable generation image editing video generation audio generation 3D generation motion generation semantic segmentation depth estimation IMAGINE-E structured output generation realism consistency specific domain generation challenging scenario generation multi-style creation tasks