Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with
  Multimodal Large Language Model
The rapid expansion of mobile internet has resulted in a substantial increase
in user-generated content (UGC) images, thereby making the thorough assessment
of UGC images both urgent and essential. Recently, multimodal large language
models (MLLMs) have shown great potential in image quality assessment (IQA) and
image aesthetic assessment (IAA). Despite this progress, effectively scoring
the quality and aesthetics of UGC images still faces two main challenges: 1) A
single score is inadequate to capture the hierarchical human perception. 2) How
to use MLLMs to output numerical scores, such as mean opinion scores (MOS),
remains an open question. To address these challenges, we introduce a novel
dataset, named Realistic image Quality and Aesthetic (RealQA), including 14,715
UGC images, each of which is annoted with 10 fine-grained attributes. These
attributes span three levels: low level (e.g., image clarity), middle level
(e.g., subject integrity) and high level (e.g., composition). Besides, we
conduct a series of in-depth and comprehensive investigations into how to
effectively predict numerical scores using MLLMs. Surprisingly, by predicting
just two extra significant digits, the next token paradigm can achieve SOTA
performance. Furthermore, with the help of chain of thought (CoT) combined with
the learnt fine-grained attributes, the proposed method can outperform SOTA
methods on five public datasets for IQA and IAA with superior interpretability
and show strong zero-shot generalization for video quality assessment (VQA).
The code and dataset will be released.
67cf981ec8f2a661dc93a6fd
Mingxing Li
claimed_verified
2025-03-11T08:23:00.614Z
6513a0f14f1682e4407758a9
/avatars/b2a6886114492944cfa235363817565f.svg
Mingxing Li
user
MingxingLi
67cf981ec8f2a661dc93a6fe
Rui Wang
67cf981ec8f2a661dc93a6ff
Lei Sun
67cf981ec8f2a661dc93a700
Yancheng Bai
67cf981ec8f2a661dc93a701
Xiangxiang Chu
Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with
  Multimodal Large Language Model
The rapid expansion of mobile internet has resulted in a substantial increase
in user-generated content (UGC) images, thereby making the thorough assessment
of UGC images both urgent and essential. Recently, multimodal large language
models (MLLMs) have shown great potential in image quality assessment (IQA) and
image aesthetic assessment (IAA). Despite this progress, effectively scoring
the quality and aesthetics of UGC images still faces two main challenges: 1) A
single score is inadequate to capture the hierarchical human perception. 2) How
to use MLLMs to output numerical scores, such as mean opinion scores (MOS),
remains an open question. To address these challenges, we introduce a novel
dataset, named Realistic image Quality and Aesthetic (RealQA), including 14,715
UGC images, each of which is annoted with 10 fine-grained attributes. These
attributes span three levels: low level (e.g., image clarity), middle level
(e.g., subject integrity) and high level (e.g., composition). Besides, we
conduct a series of in-depth and comprehensive investigations into how to
effectively predict numerical scores using MLLMs. Surprisingly, by predicting
just two extra significant digits, the next token paradigm can achieve SOTA
performance. Furthermore, with the help of chain of thought (CoT) combined with
the learnt fine-grained attributes, the proposed method can outperform SOTA
methods on five public datasets for IQA and IAA with superior interpretability
and show strong zero-shot generalization for video quality assessment (VQA).
The code and dataset will be released.
The introduction of RealQA dataset and the use of MLLMs with chain of thought for predicting image quality and aesthetic scores achieves state-of-the-art performance and zero-shot generalization for video quality assessment.
multimodal large language models image quality assessment image aesthetic assessment next token paradigm chain of thought video quality assessment