title: On Path to Multimodal Generalist: General-Level and General-Bench
summary: The Multimodal Large Language Model (MLLM) is currently experiencing rapid
  growth, driven by the advanced capabilities of LLMs. Unlike earlier
  specialists, existing MLLMs are evolving towards a Multimodal Generalist
  paradigm. Initially limited to understanding multiple modalities, these models
  have advanced to not only comprehend but also generate across modalities. Their
  capabilities have expanded from coarse-grained to fine-grained multimodal
  understanding and from supporting limited modalities to arbitrary ones. While
  many benchmarks exist to assess MLLMs, a critical question arises: Can we
  simply assume that higher performance across tasks indicates a stronger MLLM
  capability, bringing us closer to human-level AI? We argue that the answer is
  not as straightforward as it seems. This project introduces General-Level, an
  evaluation framework that defines 5-scale levels of MLLM performance and
  generality, offering a methodology to compare MLLMs and gauge the progress of
  existing systems towards more robust multimodal generalists and, ultimately,
  towards AGI. At the core of the framework is the concept of Synergy, which
  measures whether models maintain consistent capabilities across comprehension
  and generation, and across multiple modalities. To support this evaluation, we
  present General-Bench, which encompasses a broader spectrum of skills,
  modalities, formats, and capabilities, including over 700 tasks and 325,800
  instances. The evaluation results that involve over 100 existing
  state-of-the-art MLLMs uncover the capability rankings of generalists,
  highlighting the challenges in reaching genuine AI. We expect this project to
  pave the way for future research on next-generation multimodal foundation
  models, providing a robust infrastructure to accelerate the realization of AGI.
  Project page: https://generalist.top/
paper_authors._id: 681c6c1817fc8222eff39a1a
paper_authors.name: Hao Fei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T09:58:07.591Z
paper_authors._id: 647773a1168cb428e00e9a8f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647773a1168cb428e00e9a8f/NiRR3ScY6Plzjibfwy1hC.jpeg
paper_authors.fullname: Hao Fei
paper_authors.type: user
paper_authors.user: scofield7419
paper_authors._id: 681c6c1817fc8222eff39a1b
paper_authors.name: Yuan Zhou
paper_authors._id: 681c6c1817fc8222eff39a1c
paper_authors.name: Juncheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:36:52.461Z
paper_authors._id: 67bc247b593452cc18965cb1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/EA3kTYaaff0Hr7-dGiOOj.png
paper_authors.fullname: JUNCHENG LI
paper_authors.type: user
paper_authors.user: JunchengLi
paper_authors._id: 681c6c1817fc8222eff39a1d
paper_authors.name: Xiangtai Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:36:59.117Z
paper_authors._id: 63958b4414513eaf9029ebf1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
paper_authors.fullname: Xiangtai Li
paper_authors.type: user
paper_authors.user: LXT
paper_authors._id: 681c6c1817fc8222eff39a1e
paper_authors.name: Qingshan Xu
paper_authors._id: 681c6c1817fc8222eff39a1f
paper_authors.name: Bobo Li
paper_authors._id: 681c6c1817fc8222eff39a20
paper_authors.name: Shengqiong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:22:39.333Z
paper_authors._id: 64c139d867eff857ea51caa8
paper_authors.avatarUrl: /avatars/4b7b3f41c2e2cfa21dd43bbac6e081ae.svg
paper_authors.fullname: Shengqiong Wu
paper_authors.type: user
paper_authors.user: ChocoWu
paper_authors._id: 681c6c1817fc8222eff39a21
paper_authors.name: Yaoting Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:37:27.790Z
paper_authors._id: 64ff369d9abcc85a5519b33e
paper_authors.avatarUrl: /avatars/4b99cdaf5f970d930b196eddf1e5e499.svg
paper_authors.fullname: Yaoting Wang
paper_authors.type: user
paper_authors.user: Gh0stAR
paper_authors._id: 681c6c1817fc8222eff39a22
paper_authors.name: Junbao Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:37:34.046Z
paper_authors._id: 67e906836c7216f5bf91f70c
paper_authors.avatarUrl: /avatars/9c7f34d5b1d41ad7231d2733a399abb3.svg
paper_authors.fullname: junbao.zhou
paper_authors.type: user
paper_authors.user: junbaozhou
paper_authors._id: 681c6c1817fc8222eff39a23
paper_authors.name: Jiahao Meng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:37:40.200Z
paper_authors._id: 65a28e129acab19980226731
paper_authors.avatarUrl: /avatars/abc3828f807efc4e03837b0eae063f98.svg
paper_authors.fullname: Jiahao Meng
paper_authors.type: user
paper_authors.user: marinero4972
paper_authors._id: 681c6c1817fc8222eff39a24
paper_authors.name: Qingyu Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:22:34.088Z
paper_authors._id: 656724074f6ec72017754d33
paper_authors.avatarUrl: /avatars/e61de248f6f53719b2375077340dd033.svg
paper_authors.fullname: QingyuShi
paper_authors.type: user
paper_authors.user: QingyuShi
paper_authors._id: 681c6c1817fc8222eff39a25
paper_authors.name: Zhiyuan Zhou
paper_authors._id: 681c6c1817fc8222eff39a26
paper_authors.name: Liangtao Shi
paper_authors._id: 681c6c1817fc8222eff39a27
paper_authors.name: Minghe Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:38:16.554Z
paper_authors._id: 648ef24dc92367eecac0f4bd
paper_authors.avatarUrl: /avatars/38f1afd6b52efeee3aa41cc80225d788.svg
paper_authors.fullname: Minghe Gao
paper_authors.type: user
paper_authors.user: gmh5811
paper_authors._id: 681c6c1817fc8222eff39a28
paper_authors.name: Daoan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:38:28.567Z
paper_authors._id: 6241b95cfee9374a2598ecfe
paper_authors.avatarUrl: /avatars/196669df1689a5872fc18b271e80fdc1.svg
paper_authors.fullname: Zhang Daoan
paper_authors.type: user
paper_authors.user: hazard
paper_authors._id: 681c6c1817fc8222eff39a29
paper_authors.name: Zhiqi Ge
paper_authors._id: 681c6c1817fc8222eff39a2a
paper_authors.name: Weiming Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T10:36:05.177Z
paper_authors._id: 67ebcf69f5870fc1da715eda
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/TbfMOG2cKwPEh2xQdIqbr.png
paper_authors.fullname: Weiming Wu
paper_authors.type: user
paper_authors.user: wwm1415
paper_authors._id: 681c6c1817fc8222eff39a2b
paper_authors.name: Siliang Tang
paper_authors._id: 681c6c1817fc8222eff39a2c
paper_authors.name: Kaihang Pan
paper_authors._id: 681c6c1817fc8222eff39a2d
paper_authors.name: Yaobo Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:55:06.463Z
paper_authors._id: 662917afda1cae6cbb50cd00
paper_authors.avatarUrl: /avatars/aa66de6cef6665c5d67071d82bac35c4.svg
paper_authors.fullname: Yaobo Ye
paper_authors.type: user
paper_authors.user: superyyb
paper_authors._id: 681c6c1817fc8222eff39a2e
paper_authors.name: Haobo Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:39:11.094Z
paper_authors._id: 6391e41f2e73987364e6bcb2
paper_authors.avatarUrl: /avatars/d09a9ee329bb8c3a9e2929d67d24e97d.svg
paper_authors.fullname: Haobo Yuan
paper_authors.type: user
paper_authors.user: HarborYuan
paper_authors._id: 681c6c1817fc8222eff39a2f
paper_authors.name: Tao Zhang
paper_authors._id: 681c6c1817fc8222eff39a30
paper_authors.name: Tianjie Ju
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:53:06.930Z
paper_authors._id: 6816d98fc075e49c1b15928e
paper_authors.avatarUrl: /avatars/6b24d047fc25075bedb3e74f78981bc0.svg
paper_authors.fullname: Tianjie Ju
paper_authors.type: user
paper_authors.user: jometeorieNUS
paper_authors._id: 681c6c1817fc8222eff39a31
paper_authors.name: Zixiang Meng
paper_authors._id: 681c6c1817fc8222eff39a32
paper_authors.name: Shilin Xu
paper_authors._id: 681c6c1817fc8222eff39a33
paper_authors.name: Liyu Jia
paper_authors._id: 681c6c1817fc8222eff39a34
paper_authors.name: Wentao Hu
paper_authors._id: 681c6c1817fc8222eff39a35
paper_authors.name: Meng Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:22:37.235Z
paper_authors._id: 64ad1c0bad6218d51a07b54e
paper_authors.avatarUrl: /avatars/0f84d9a51c6ca9bcef44de2d7c707d9b.svg
paper_authors.fullname: LUO MENG
paper_authors.type: user
paper_authors.user: Eureka-Leo
paper_authors._id: 681c6c1817fc8222eff39a36
paper_authors.name: Jiebo Luo
paper_authors._id: 681c6c1817fc8222eff39a37
paper_authors.name: Tat-Seng Chua
paper_authors._id: 681c6c1817fc8222eff39a38
paper_authors.name: Shuicheng Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T08:52:30.205Z
paper_authors._id: 67eaa070b9fa8908e151fd7d
paper_authors.avatarUrl: /avatars/1fe2fd678d2e71099a83a9bcb9ab517e.svg
paper_authors.fullname: shuicheng yan
paper_authors.type: user
paper_authors.user: shuicheng
paper_authors._id: 681c6c1817fc8222eff39a39
paper_authors.name: Hanwang Zhang
paper_title: On Path to Multimodal Generalist: General-Level and General-Bench
paper_summary: The Multimodal Large Language Model (MLLM) is currently experiencing rapid
  growth, driven by the advanced capabilities of LLMs. Unlike earlier
  specialists, existing MLLMs are evolving towards a Multimodal Generalist
  paradigm. Initially limited to understanding multiple modalities, these models
  have advanced to not only comprehend but also generate across modalities. Their
  capabilities have expanded from coarse-grained to fine-grained multimodal
  understanding and from supporting limited modalities to arbitrary ones. While
  many benchmarks exist to assess MLLMs, a critical question arises: Can we
  simply assume that higher performance across tasks indicates a stronger MLLM
  capability, bringing us closer to human-level AI? We argue that the answer is
  not as straightforward as it seems. This project introduces General-Level, an
  evaluation framework that defines 5-scale levels of MLLM performance and
  generality, offering a methodology to compare MLLMs and gauge the progress of
  existing systems towards more robust multimodal generalists and, ultimately,
  towards AGI. At the core of the framework is the concept of Synergy, which
  measures whether models maintain consistent capabilities across comprehension
  and generation, and across multiple modalities. To support this evaluation, we
  present General-Bench, which encompasses a broader spectrum of skills,
  modalities, formats, and capabilities, including over 700 tasks and 325,800
  instances. The evaluation results that involve over 100 existing
  state-of-the-art MLLMs uncover the capability rankings of generalists,
  highlighting the challenges in reaching genuine AI. We expect this project to
  pave the way for future research on next-generation multimodal foundation
  models, providing a robust infrastructure to accelerate the realization of AGI.
  Project page: https://generalist.top/
paper_ai_summary: A General-Level evaluation framework is introduced to assess multimodal generalist performance in LLMs, using Synergy to measure consistent capabilities across comprehension and generation.
paper_ai_keywords: Multimodal Large Language Model Multimodal Generalist paradigm coarse-grained multimodal understanding fine-grained multimodal understanding General-Level evaluation framework Synergy General-Bench next-generation multimodal foundation models