On Path to Multimodal Generalist: General-Level and General-Bench
The Multimodal Large Language Model (MLLM) is currently experiencing rapid
growth, driven by the advanced capabilities of LLMs. Unlike earlier
specialists, existing MLLMs are evolving towards a Multimodal Generalist
paradigm. Initially limited to understanding multiple modalities, these models
have advanced to not only comprehend but also generate across modalities. Their
capabilities have expanded from coarse-grained to fine-grained multimodal
understanding and from supporting limited modalities to arbitrary ones. While
many benchmarks exist to assess MLLMs, a critical question arises: Can we
simply assume that higher performance across tasks indicates a stronger MLLM
capability, bringing us closer to human-level AI? We argue that the answer is
not as straightforward as it seems. This project introduces General-Level, an
evaluation framework that defines 5-scale levels of MLLM performance and
generality, offering a methodology to compare MLLMs and gauge the progress of
existing systems towards more robust multimodal generalists and, ultimately,
towards AGI. At the core of the framework is the concept of Synergy, which
measures whether models maintain consistent capabilities across comprehension
and generation, and across multiple modalities. To support this evaluation, we
present General-Bench, which encompasses a broader spectrum of skills,
modalities, formats, and capabilities, including over 700 tasks and 325,800
instances. The evaluation results that involve over 100 existing
state-of-the-art MLLMs uncover the capability rankings of generalists,
highlighting the challenges in reaching genuine AI. We expect this project to
pave the way for future research on next-generation multimodal foundation
models, providing a robust infrastructure to accelerate the realization of AGI.
Project page: https://generalist.top/
681c6c1817fc8222eff39a1a
Hao Fei
claimed_verified
2025-05-08T09:58:07.591Z
647773a1168cb428e00e9a8f
https://cdn-avatars.huggingface.co/v1/production/uploads/647773a1168cb428e00e9a8f/NiRR3ScY6Plzjibfwy1hC.jpeg
Hao Fei
user
scofield7419
681c6c1817fc8222eff39a1b
Yuan Zhou
681c6c1817fc8222eff39a1c
Juncheng Li
admin_assigned
2025-05-09T08:36:52.461Z
67bc247b593452cc18965cb1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/EA3kTYaaff0Hr7-dGiOOj.png
JUNCHENG LI
user
JunchengLi
681c6c1817fc8222eff39a1d
Xiangtai Li
admin_assigned
2025-05-09T08:36:59.117Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
681c6c1817fc8222eff39a1e
Qingshan Xu
681c6c1817fc8222eff39a1f
Bobo Li
681c6c1817fc8222eff39a20
Shengqiong Wu
claimed_verified
2025-05-09T07:22:39.333Z
64c139d867eff857ea51caa8
/avatars/4b7b3f41c2e2cfa21dd43bbac6e081ae.svg
Shengqiong Wu
user
ChocoWu
681c6c1817fc8222eff39a21
Yaoting Wang
admin_assigned
2025-05-09T08:37:27.790Z
64ff369d9abcc85a5519b33e
/avatars/4b99cdaf5f970d930b196eddf1e5e499.svg
Yaoting Wang
user
Gh0stAR
681c6c1817fc8222eff39a22
Junbao Zhou
admin_assigned
2025-05-09T08:37:34.046Z
67e906836c7216f5bf91f70c
/avatars/9c7f34d5b1d41ad7231d2733a399abb3.svg
junbao.zhou
user
junbaozhou
681c6c1817fc8222eff39a23
Jiahao Meng
admin_assigned
2025-05-09T08:37:40.200Z
65a28e129acab19980226731
/avatars/abc3828f807efc4e03837b0eae063f98.svg
Jiahao Meng
user
marinero4972
681c6c1817fc8222eff39a24
Qingyu Shi
claimed_verified
2025-05-09T07:22:34.088Z
656724074f6ec72017754d33
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zLIgt4xvfbYPLZh0E3WWF.png
QingyuShi
user
QingyuShi
681c6c1817fc8222eff39a25
Zhiyuan Zhou
681c6c1817fc8222eff39a26
Liangtao Shi
681c6c1817fc8222eff39a27
Minghe Gao
admin_assigned
2025-05-09T08:38:16.554Z
648ef24dc92367eecac0f4bd
/avatars/38f1afd6b52efeee3aa41cc80225d788.svg
Minghe Gao
user
gmh5811
681c6c1817fc8222eff39a28
Daoan Zhang
admin_assigned
2025-05-09T08:38:28.567Z
6241b95cfee9374a2598ecfe
/avatars/196669df1689a5872fc18b271e80fdc1.svg
Zhang Daoan
user
hazard
681c6c1817fc8222eff39a29
Zhiqi Ge
681c6c1817fc8222eff39a2a
Weiming Wu
claimed_verified
2025-05-09T10:36:05.177Z
67ebcf69f5870fc1da715eda
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/TbfMOG2cKwPEh2xQdIqbr.png
Weiming Wu
user
wwm1415
681c6c1817fc8222eff39a2b
Siliang Tang
681c6c1817fc8222eff39a2c
Kaihang Pan
681c6c1817fc8222eff39a2d
Yaobo Ye
admin_assigned
2025-05-09T08:55:06.463Z
662917afda1cae6cbb50cd00
/avatars/aa66de6cef6665c5d67071d82bac35c4.svg
Yaobo Ye
user
superyyb
681c6c1817fc8222eff39a2e
Haobo Yuan
admin_assigned
2025-05-09T08:39:11.094Z
6391e41f2e73987364e6bcb2
/avatars/d09a9ee329bb8c3a9e2929d67d24e97d.svg
Haobo Yuan
user
HarborYuan
681c6c1817fc8222eff39a2f
Tao Zhang
681c6c1817fc8222eff39a30
Tianjie Ju
admin_assigned
2025-05-09T08:53:06.930Z
6816d98fc075e49c1b15928e
/avatars/6b24d047fc25075bedb3e74f78981bc0.svg
Tianjie Ju
user
jometeorieNUS
681c6c1817fc8222eff39a31
Zixiang Meng
681c6c1817fc8222eff39a32
Shilin Xu
681c6c1817fc8222eff39a33
Liyu Jia
681c6c1817fc8222eff39a34
Wentao Hu
681c6c1817fc8222eff39a35
Meng Luo
claimed_verified
2025-05-09T07:22:37.235Z
64ad1c0bad6218d51a07b54e
/avatars/0f84d9a51c6ca9bcef44de2d7c707d9b.svg
LUO MENG
user
Eureka-Leo
681c6c1817fc8222eff39a36
Jiebo Luo
681c6c1817fc8222eff39a37
Tat-Seng Chua
681c6c1817fc8222eff39a38
Shuicheng Yan
admin_assigned
2025-05-09T08:52:30.205Z
67eaa070b9fa8908e151fd7d
/avatars/1fe2fd678d2e71099a83a9bcb9ab517e.svg
shuicheng yan
user
shuicheng
681c6c1817fc8222eff39a39
Hanwang Zhang
On Path to Multimodal Generalist: General-Level and General-Bench
The Multimodal Large Language Model (MLLM) is currently experiencing rapid
growth, driven by the advanced capabilities of LLMs. Unlike earlier
specialists, existing MLLMs are evolving towards a Multimodal Generalist
paradigm. Initially limited to understanding multiple modalities, these models
have advanced to not only comprehend but also generate across modalities. Their
capabilities have expanded from coarse-grained to fine-grained multimodal
understanding and from supporting limited modalities to arbitrary ones. While
many benchmarks exist to assess MLLMs, a critical question arises: Can we
simply assume that higher performance across tasks indicates a stronger MLLM
capability, bringing us closer to human-level AI? We argue that the answer is
not as straightforward as it seems. This project introduces General-Level, an
evaluation framework that defines 5-scale levels of MLLM performance and
generality, offering a methodology to compare MLLMs and gauge the progress of
existing systems towards more robust multimodal generalists and, ultimately,
towards AGI. At the core of the framework is the concept of Synergy, which
measures whether models maintain consistent capabilities across comprehension
and generation, and across multiple modalities. To support this evaluation, we
present General-Bench, which encompasses a broader spectrum of skills,
modalities, formats, and capabilities, including over 700 tasks and 325,800
instances. The evaluation results that involve over 100 existing
state-of-the-art MLLMs uncover the capability rankings of generalists,
highlighting the challenges in reaching genuine AI. We expect this project to
pave the way for future research on next-generation multimodal foundation
models, providing a robust infrastructure to accelerate the realization of AGI.
Project page: https://generalist.top/
A General-Level evaluation framework is introduced to assess multimodal generalist performance in LLMs, using Synergy to measure consistent capabilities across comprehension and generation.
Multimodal Large Language Model Multimodal Generalist paradigm coarse-grained multimodal understanding fine-grained multimodal understanding General-Level evaluation framework Synergy General-Bench next-generation multimodal foundation models