VLRewardBench: A Challenging Benchmark for Vision-Language Generative
  Reward Models
Vision-language generative reward models (VL-GenRMs) play a crucial role in
aligning and evaluating multimodal AI systems, yet their own evaluation remains
under-explored. Current assessment methods primarily rely on AI-annotated
preference labels from traditional VL tasks, which can introduce biases and
often fail to effectively challenge state-of-the-art models. To address these
limitations, we introduce VL-RewardBench, a comprehensive benchmark spanning
general multimodal queries, visual hallucination detection, and complex
reasoning tasks. Through our AI-assisted annotation pipeline combining sample
selection with human verification, we curate 1,250 high-quality examples
specifically designed to probe model limitations. Comprehensive evaluation
across 16 leading large vision-language models, demonstrates VL-RewardBench's
effectiveness as a challenging testbed, where even GPT-4o achieves only 65.4%
accuracy, and state-of-the-art open-source models such as Qwen2-VL-72B,
struggle to surpass random-guessing. Importantly, performance on VL-RewardBench
strongly correlates (Pearson's r > 0.9) with MMMU-Pro accuracy using Best-of-N
sampling with VL-GenRMs. Analysis experiments uncover three critical insights
for improving VL-GenRMs: (i) models predominantly fail at basic visual
perception tasks rather than reasoning tasks; (ii) inference-time scaling
benefits vary dramatically by model capacity; and (iii) training VL-GenRMs to
learn to judge substantially boosts judgment capability (+14.7% accuracy for a
7B VL-GenRM). We believe VL-RewardBench along with the experimental insights
will become a valuable resource for advancing VL-GenRMs.
6746f9c205afcd8837d520b7
Lei Li
claimed_verified
2024-12-09T14:47:54.952Z
6038d6d0612f5eef3cc05ea9
https://cdn-avatars.huggingface.co/v1/production/uploads/6038d6d0612f5eef3cc05ea9/ryhvAX5djQpD5OrIlZQ1f.jpeg
Lei Li
user
tobiaslee
6746f9c205afcd8837d520b8
Yuancheng Wei
6746f9c205afcd8837d520b9
Zhihui Xie
claimed_verified
2024-11-28T12:19:47.072Z
622f103fc78da4c7ebd7c887
/avatars/b0c7cd29835d92c2cd584947fcd5d520.svg
Xie
user
Zhihui
6746f9c205afcd8837d520ba
Xuqing Yang
6746f9c205afcd8837d520bb
Yifan Song
6746f9c205afcd8837d520bc
Peiyi Wang
6746f9c205afcd8837d520bd
Chenxin An
6746f9c205afcd8837d520be
Tianyu Liu
6746f9c205afcd8837d520bf
Sujian Li
6746f9c205afcd8837d520c0
Bill Yuchen Lin
6746f9c205afcd8837d520c1
Lingpeng Kong
6746f9c205afcd8837d520c2
Qi Liu
VLRewardBench: A Challenging Benchmark for Vision-Language Generative
  Reward Models
Vision-language generative reward models (VL-GenRMs) play a crucial role in
aligning and evaluating multimodal AI systems, yet their own evaluation remains
under-explored. Current assessment methods primarily rely on AI-annotated
preference labels from traditional VL tasks, which can introduce biases and
often fail to effectively challenge state-of-the-art models. To address these
limitations, we introduce VL-RewardBench, a comprehensive benchmark spanning
general multimodal queries, visual hallucination detection, and complex
reasoning tasks. Through our AI-assisted annotation pipeline combining sample
selection with human verification, we curate 1,250 high-quality examples
specifically designed to probe model limitations. Comprehensive evaluation
across 16 leading large vision-language models, demonstrates VL-RewardBench's
effectiveness as a challenging testbed, where even GPT-4o achieves only 65.4%
accuracy, and state-of-the-art open-source models such as Qwen2-VL-72B,
struggle to surpass random-guessing. Importantly, performance on VL-RewardBench
strongly correlates (Pearson's r > 0.9) with MMMU-Pro accuracy using Best-of-N
sampling with VL-GenRMs. Analysis experiments uncover three critical insights
for improving VL-GenRMs: (i) models predominantly fail at basic visual
perception tasks rather than reasoning tasks; (ii) inference-time scaling
benefits vary dramatically by model capacity; and (iii) training VL-GenRMs to
learn to judge substantially boosts judgment capability (+14.7% accuracy for a
7B VL-GenRM). We believe VL-RewardBench along with the experimental insights
will become a valuable resource for advancing VL-GenRMs.
VL-RewardBench is a comprehensive benchmark designed to challenge vision-language generative reward models across various tasks, demonstrating their limitations and providing insights for improvement.
vision-language generative reward models multimodal AI systems AI-annotated preference labels visual hallucination detection complex reasoning tasks AI-assisted annotation pipeline human verification large vision-language models GPT-4o Qwen2-VL-72B MMMU-Pro Best-of-N sampling inference-time scaling training VL-GenRMs to learn to judge