Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability
shall be elicited to evaluate the aesthetics of artworks. To facilitate this
investigation, we construct MM-StyleBench, a novel high-quality dataset for
benchmarking artistic stylization. We then develop a principled method for
human preference modeling and perform a systematic correlation analysis between
MLLMs' responses and human preference. Our experiments reveal an inherent
hallucination issue of MLLMs in art evaluation, associated with response
subjectivity. ArtCoT is proposed, demonstrating that art-specific task
decomposition and the use of concrete language boost MLLMs' reasoning ability
for aesthetics. Our findings offer valuable insights into MLLMs for art and can
benefit a wide range of downstream applications, such as style transfer and
artistic image generation. Code available at
https://github.com/songrise/MLLM4Art.
6788a30ee9e04d1c80fb1d6d
Ruixiang Jiang
admin_assigned
2025-01-16T08:54:16.366Z
6351382f40dffad651ef3fbd
/avatars/3ac2de7c49086bb37cc4f4bd29ed72f2.svg
JIANG
user
Ruixiang
6788a30ee9e04d1c80fb1d6e
Changwen Chen
admin_assigned
2025-01-16T08:54:22.754Z
64f95c12ec913b4f977ba028
/avatars/b128850ee2a3667d0b1972659cd5f0ae.svg
Chang Wen Cheng
user
Vincentchang
Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability
shall be elicited to evaluate the aesthetics of artworks. To facilitate this
investigation, we construct MM-StyleBench, a novel high-quality dataset for
benchmarking artistic stylization. We then develop a principled method for
human preference modeling and perform a systematic correlation analysis between
MLLMs' responses and human preference. Our experiments reveal an inherent
hallucination issue of MLLMs in art evaluation, associated with response
subjectivity. ArtCoT is proposed, demonstrating that art-specific task
decomposition and the use of concrete language boost MLLMs' reasoning ability
for aesthetics. Our findings offer valuable insights into MLLMs for art and can
benefit a wide range of downstream applications, such as style transfer and
artistic image generation. Code available at
https://github.com/songrise/MLLM4Art.
A study on Multimodal LLMs for evaluating artwork aesthetics using MM-StyleBench shows that task decomposition and concrete language enhance MLLMs' reasoning ability despite hallucination issues.
Multimodal LLMs MLLMs MM-StyleBench human preference modeling correlation analysis response subjectivity art-specific task decomposition concrete language ArtCoT style transfer artistic image generation