title: ViLBench: A Suite for Vision-Language Process Reward Modeling
summary: Process-supervised reward models serve as a fine-grained function that
  provides detailed step-wise feedback to model responses, facilitating effective
  selection of reasoning trajectories for complex tasks. Despite its advantages,
  evaluation on PRMs remains less explored, especially in the multimodal domain.
  To address this gap, this paper first benchmarks current vision large language
  models (VLLMs) as two types of reward models: output reward models (ORMs) and
  process reward models (PRMs) on multiple vision-language benchmarks, which
  reveal that neither ORM nor PRM consistently outperforms across all tasks, and
  superior VLLMs do not necessarily yield better rewarding performance. To
  further advance evaluation, we introduce ViLBench, a vision-language benchmark
  designed to require intensive process reward signals. Notably, OpenAI's GPT-4o
  with Chain-of-Thought (CoT) achieves only 27.3% accuracy, indicating the
  benchmark's challenge for current VLLMs. Lastly, we preliminarily showcase a
  promising pathway towards bridging the gap between general VLLMs and reward
  models -- by collecting 73.6K vision-language process reward data using an
  enhanced tree-search algorithm, our 3B model is able to achieve an average
  improvement of 3.3% over standard CoT and up to 2.5% compared to its untrained
  counterpart on ViLBench by selecting OpenAI o1's generations. We release the
  implementations at https://ucsc-vlaa.github.io/ViLBench with our code, model,
  and data.
paper_authors._id: 67e4dc6a38e4d1444c71ce70
paper_authors.name: Haoqin Tu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:17:04.325Z
paper_authors._id: 604ae011caabafacfa48e3de
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1615519738679-noauth.jpeg
paper_authors.fullname: Haoqin Tu
paper_authors.type: user
paper_authors.user: PahaII
paper_authors._id: 67e4dc6a38e4d1444c71ce71
paper_authors.name: Weitao Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:17:11.346Z
paper_authors._id: 6625e4e960286184cd2a3cc8
paper_authors.avatarUrl: /avatars/894c4ba1dbf776c5db62e03cde07f9f8.svg
paper_authors.fullname: Weitao Feng
paper_authors.type: user
paper_authors.user: Helicopt
paper_authors._id: 67e4dc6a38e4d1444c71ce72
paper_authors.name: Hardy Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:17:20.107Z
paper_authors._id: 65ca20731082be5463d0945c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xpySHGNH7_yyH58gUyvvZ.jpeg
paper_authors.fullname: Hardy Chen
paper_authors.type: user
paper_authors.user: alihiker
paper_authors._id: 67e4dc6a38e4d1444c71ce73
paper_authors.name: Hui Liu
paper_authors._id: 67e4dc6a38e4d1444c71ce74
paper_authors.name: Xianfeng Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:17:40.389Z
paper_authors._id: 6465f6467ff8fcbef7d22513
paper_authors.avatarUrl: /avatars/07992835c235fbb07016a0ea4f1d61cb.svg
paper_authors.fullname: Xianfeng Tang
paper_authors.type: user
paper_authors.user: xianft
paper_authors._id: 67e4dc6a38e4d1444c71ce75
paper_authors.name: Cihang Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:17:34.681Z
paper_authors._id: 645eb61da3c5cd8a16efffff
paper_authors.avatarUrl: /avatars/9112bfeed598dfabf9e077e69e09ecc9.svg
paper_authors.fullname: Cihang Xie
paper_authors.type: user
paper_authors.user: cihangxie
paper_title: ViLBench: A Suite for Vision-Language Process Reward Modeling
paper_summary: Process-supervised reward models serve as a fine-grained function that
  provides detailed step-wise feedback to model responses, facilitating effective
  selection of reasoning trajectories for complex tasks. Despite its advantages,
  evaluation on PRMs remains less explored, especially in the multimodal domain.
  To address this gap, this paper first benchmarks current vision large language
  models (VLLMs) as two types of reward models: output reward models (ORMs) and
  process reward models (PRMs) on multiple vision-language benchmarks, which
  reveal that neither ORM nor PRM consistently outperforms across all tasks, and
  superior VLLMs do not necessarily yield better rewarding performance. To
  further advance evaluation, we introduce ViLBench, a vision-language benchmark
  designed to require intensive process reward signals. Notably, OpenAI's GPT-4o
  with Chain-of-Thought (CoT) achieves only 27.3% accuracy, indicating the
  benchmark's challenge for current VLLMs. Lastly, we preliminarily showcase a
  promising pathway towards bridging the gap between general VLLMs and reward
  models -- by collecting 73.6K vision-language process reward data using an
  enhanced tree-search algorithm, our 3B model is able to achieve an average
  improvement of 3.3% over standard CoT and up to 2.5% compared to its untrained
  counterpart on ViLBench by selecting OpenAI o1's generations. We release the
  implementations at https://ucsc-vlaa.github.io/ViLBench with our code, model,
  and data.
paper_ai_summary: Evaluation of process-supervised reward models on vision-language benchmarks reveals variability in performance and introduces a challenging benchmark, ViLBench, where current models show significant room for improvement.
paper_ai_keywords: process-supervised reward models VLLMs output reward models process reward models vision-language benchmarks Chain-of-Thought ViLBench