MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for
  Text-to-Image Generation?
While text-to-image models like DALLE-3 and Stable Diffusion are rapidly
proliferating, they often encounter challenges such as hallucination, bias, and
the production of unsafe, low-quality output. To effectively address these
issues, it is crucial to align these models with desired behaviors based on
feedback from a multimodal judge. Despite their significance, current
multimodal judges frequently undergo inadequate evaluation of their
capabilities and limitations, potentially leading to misalignment and unsafe
fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel
benchmark which incorporates a comprehensive preference dataset to evaluate
multimodal judges in providing feedback for image generation models across four
key perspectives: alignment, safety, image quality, and bias. Specifically, we
evaluate a large variety of multimodal judges including smaller-sized
CLIP-based scoring models, open-source VLMs (e.g. LLaVA family), and
close-source VLMs (e.g. GPT-4o, Claude 3) on each decomposed subcategory of our
preference dataset. Experiments reveal that close-source VLMs generally provide
better feedback, with GPT-4o outperforming other judges in average. Compared
with open-source VLMs, smaller-sized scoring models can provide better feedback
regarding text-image alignment and image quality, while VLMs provide more
accurate feedback regarding safety and generation bias due to their stronger
reasoning capabilities. Further studies in feedback scale reveal that VLM
judges can generally provide more accurate and stable feedback in natural
language (Likert-scale) than numerical scales. Notably, human evaluations on
end-to-end fine-tuned models using separate feedback from these multimodal
judges provide similar conclusions, further confirming the effectiveness of
MJ-Bench. All data, code, models are available at
https://huggingface.co/MJ-Bench.
668cbe15bddba3c354e0e7bc
Zhaorun Chen
claimed_verified
2024-07-09T07:41:52.006Z
64d448d3d095077728f7c740
https://cdn-avatars.huggingface.co/v1/production/uploads/64d448d3d095077728f7c740/4CFkiVoXXDsu7uQPTU0n3.jpeg
Zhaorun Chen
user
Zhaorun
668cbe15bddba3c354e0e7bd
Yichao Du
claimed_verified
2024-07-09T07:41:59.325Z
63e1d3451e5a4f34b7a728ef
https://cdn-avatars.huggingface.co/v1/production/uploads/63e1d3451e5a4f34b7a728ef/kHE5JHF9iZJOG0uBkvJr-.jpeg
yichaodu
user
yichaodu
668cbe15bddba3c354e0e7be
Zichen Wen
admin_assigned
2024-07-09T08:34:41.704Z
653b8c3e97a4d71d950e2f20
/avatars/b68880022e14556d0be58c69615db3be.svg
Zichen Wen
user
zichenwen
668cbe15bddba3c354e0e7bf
Yiyang Zhou
admin_assigned
2024-07-09T08:35:14.296Z
6433722c5277e3b24ef49055
/avatars/f3c5560d500c699e452986a6a45ba3ee.svg
Yiyang Zhou
user
YiyangAiLab
668cbe15bddba3c354e0e7c0
Chenhang Cui
admin_assigned
2024-07-09T08:35:28.734Z
64a26627f09ad861c85487e0
/avatars/baefd153c59f3a3238fd6034c6daf0d4.svg
ChenhangCui
user
Chenhangcui
668cbe15bddba3c354e0e7c1
Zhenzhen Weng
admin_assigned
2024-07-09T08:35:35.672Z
630834fbf0dc38fb47c245b2
/avatars/579d083fdbfba7331b07941f0bef83a4.svg
Zhenzhen Weng
user
zzweng
668cbe15bddba3c354e0e7c2
Haoqin Tu
admin_assigned
2024-07-09T08:35:41.704Z
604ae011caabafacfa48e3de
https://cdn-avatars.huggingface.co/v1/production/uploads/1615519738679-noauth.jpeg
Haoqin Tu
user
PahaII
668cbe15bddba3c354e0e7c3
Chaoqi Wang
admin_assigned
2024-07-09T08:35:48.911Z
64fac4e92d20ced4e9f67525
/avatars/3c33cf7bac9fab07aa200e553eb832e9.svg
Chaoqi Wang
user
alecwangcq
668cbe15bddba3c354e0e7c4
Zhengwei Tong
668cbe15bddba3c354e0e7c5
Qinglan Huang
admin_assigned
2024-07-09T08:36:09.433Z
6507fac29622235d7dddc430
/avatars/0fd23bc7db683a18cac8d2cc52131840.svg
hql
user
huangqinglan
668cbe15bddba3c354e0e7c6
Canyu Chen
claimed_verified
2024-07-09T07:41:55.281Z
6483af58571c2dcfa98cae82
https://cdn-avatars.huggingface.co/v1/production/uploads/6483af58571c2dcfa98cae82/d8Vnh_EG1KxeEoNAAQXHZ.jpeg
Canyu Chen
user
canyuchen
668cbe15bddba3c354e0e7c7
Qinghao Ye
claimed_verified
2024-07-09T14:05:11.232Z
64530fc01a57e1179c1fe4c0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/0lncTpIHXn6suB0p-oSma.jpeg
QinghaoYe
user
MAGAer13
668cbe15bddba3c354e0e7c8
Zhihong Zhu
admin_assigned
2024-07-09T08:36:25.102Z
64b78e70da8017900e7d1972
/avatars/f862e7f98cfc3ffb820309ffe7bc73d3.svg
GGBond
user
PKUGGBond
668cbe15bddba3c354e0e7c9
Yuqing Zhang
claimed_verified
2024-07-09T21:09:13.885Z
668d4fde7dbe0ab21ddba8ed
/avatars/ee5671cc0020cc1b7c6847d7af7252a0.svg
Yuqing  Zhang
user
yuqingzhang
668cbe15bddba3c354e0e7ca
Jiawei Zhou
668cbe15bddba3c354e0e7cb
Zhuokai Zhao
admin_assigned
2024-07-09T08:37:03.316Z
63af25605fe9db73f67a0fb7
https://cdn-avatars.huggingface.co/v1/production/uploads/63af25605fe9db73f67a0fb7/mcQHbTdJVxi2GdHzRB9ad.jpeg
Zhuokai Zhao
user
zhuokai
668cbe15bddba3c354e0e7cc
Rafael Rafailov
admin_assigned
2024-07-09T08:37:13.962Z
64701c98d742e9ef6522598f
/avatars/df3a52bd4bc374d7cb4061f449f75c07.svg
Rafael Rafailov
user
rmrafailov
668cbe15bddba3c354e0e7cd
Chelsea Finn
admin_assigned
2024-07-09T08:37:19.483Z
64ac22a9193f0a807deb673d
/avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
Chelsea Finn
user
cbfinn
668cbe15bddba3c354e0e7ce
Huaxiu Yao
admin_assigned
2024-07-09T08:37:27.914Z
65f667a455009c4ad9e6ac4c
/avatars/ed10c2cf2ba3fde6d7da93f076961607.svg
Yao
user
Huaxiu
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for
  Text-to-Image Generation?
While text-to-image models like DALLE-3 and Stable Diffusion are rapidly
proliferating, they often encounter challenges such as hallucination, bias, and
the production of unsafe, low-quality output. To effectively address these
issues, it is crucial to align these models with desired behaviors based on
feedback from a multimodal judge. Despite their significance, current
multimodal judges frequently undergo inadequate evaluation of their
capabilities and limitations, potentially leading to misalignment and unsafe
fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel
benchmark which incorporates a comprehensive preference dataset to evaluate
multimodal judges in providing feedback for image generation models across four
key perspectives: alignment, safety, image quality, and bias. Specifically, we
evaluate a large variety of multimodal judges including smaller-sized
CLIP-based scoring models, open-source VLMs (e.g. LLaVA family), and
close-source VLMs (e.g. GPT-4o, Claude 3) on each decomposed subcategory of our
preference dataset. Experiments reveal that close-source VLMs generally provide
better feedback, with GPT-4o outperforming other judges in average. Compared
with open-source VLMs, smaller-sized scoring models can provide better feedback
regarding text-image alignment and image quality, while VLMs provide more
accurate feedback regarding safety and generation bias due to their stronger
reasoning capabilities. Further studies in feedback scale reveal that VLM
judges can generally provide more accurate and stable feedback in natural
language (Likert-scale) than numerical scales. Notably, human evaluations on
end-to-end fine-tuned models using separate feedback from these multimodal
judges provide similar conclusions, further confirming the effectiveness of
MJ-Bench. All data, code, models are available at
https://huggingface.co/MJ-Bench.
MJ-Bench evaluates multimodal judges across alignment, safety, image quality, and bias for text-to-image models and confirms close-source VLMs generally provide better feedback.
MJ-Bench multimodal judges preference dataset CLIP-based scoring models open-source VLMs close-source VLMs GPT-4o Claude 3 text-image alignment generation bias feedback scale human evaluations