title: Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
summary: We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability
  shall be elicited to evaluate the aesthetics of artworks. To facilitate this
  investigation, we construct MM-StyleBench, a novel high-quality dataset for
  benchmarking artistic stylization. We then develop a principled method for
  human preference modeling and perform a systematic correlation analysis between
  MLLMs' responses and human preference. Our experiments reveal an inherent
  hallucination issue of MLLMs in art evaluation, associated with response
  subjectivity. ArtCoT is proposed, demonstrating that art-specific task
  decomposition and the use of concrete language boost MLLMs' reasoning ability
  for aesthetics. Our findings offer valuable insights into MLLMs for art and can
  benefit a wide range of downstream applications, such as style transfer and
  artistic image generation. Code available at
  https://github.com/songrise/MLLM4Art.
paper_authors._id: 6788a30ee9e04d1c80fb1d6d
paper_authors.name: Ruixiang Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:54:16.366Z
paper_authors._id: 6351382f40dffad651ef3fbd
paper_authors.avatarUrl: /avatars/3ac2de7c49086bb37cc4f4bd29ed72f2.svg
paper_authors.fullname: JIANG
paper_authors.type: user
paper_authors.user: Ruixiang
paper_authors._id: 6788a30ee9e04d1c80fb1d6e
paper_authors.name: Changwen Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T08:54:22.754Z
paper_authors._id: 64f95c12ec913b4f977ba028
paper_authors.avatarUrl: /avatars/b128850ee2a3667d0b1972659cd5f0ae.svg
paper_authors.fullname: Chang Wen Cheng
paper_authors.type: user
paper_authors.user: Vincentchang
paper_title: Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
paper_summary: We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability
  shall be elicited to evaluate the aesthetics of artworks. To facilitate this
  investigation, we construct MM-StyleBench, a novel high-quality dataset for
  benchmarking artistic stylization. We then develop a principled method for
  human preference modeling and perform a systematic correlation analysis between
  MLLMs' responses and human preference. Our experiments reveal an inherent
  hallucination issue of MLLMs in art evaluation, associated with response
  subjectivity. ArtCoT is proposed, demonstrating that art-specific task
  decomposition and the use of concrete language boost MLLMs' reasoning ability
  for aesthetics. Our findings offer valuable insights into MLLMs for art and can
  benefit a wide range of downstream applications, such as style transfer and
  artistic image generation. Code available at
  https://github.com/songrise/MLLM4Art.
paper_ai_summary: A study on Multimodal LLMs for evaluating artwork aesthetics using MM-StyleBench shows that task decomposition and concrete language enhance MLLMs' reasoning ability despite hallucination issues.
paper_ai_keywords: Multimodal LLMs MLLMs MM-StyleBench human preference modeling correlation analysis response subjectivity art-specific task decomposition concrete language ArtCoT style transfer artistic image generation