title: Skywork-VL Reward: An Effective Reward Model for Multimodal
    Understanding and Reasoning
summary: We propose Skywork-VL Reward, a multimodal reward model that provides reward
  signals for both multimodal understanding and reasoning tasks. Our technical
  approach comprises two key components: First, we construct a large-scale
  multimodal preference dataset that covers a wide range of tasks and scenarios,
  with responses collected from both standard vision-language models (VLMs) and
  advanced VLM reasoners. Second, we design a reward model architecture based on
  Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
  fine-tuning using pairwise ranking loss on pairwise preference data.
  Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
  results on multimodal VL-RewardBench and exhibits competitive performance on
  the text-only RewardBench benchmark. Furthermore, preference data constructed
  based on our Skywork-VL Reward proves highly effective for training Mixed
  Preference Optimization (MPO), leading to significant improvements in
  multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
  a significant advancement toward general-purpose, reliable reward models for
  multimodal alignment. Our model has been publicly released to promote
  transparency and reproducibility.
paper_authors._id: 6822cf4c3b7392637edd756a
paper_authors.name: Xiaokun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T10:27:21.212Z
paper_authors._id: 62be9b5aae56e75e4d689e7c
paper_authors.avatarUrl: /avatars/6772bc09d6eeb4e86b1210481be91720.svg
paper_authors.fullname: wangxiaokun
paper_authors.type: user
paper_authors.user: shawn0wang
paper_authors._id: 6822cf4c3b7392637edd756b
paper_authors.name: Chris
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T10:27:19.259Z
paper_authors._id: 620f5a1c3f76c50e6458a9b6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg
paper_authors.fullname: Peiyu Wang
paper_authors.type: user
paper_authors.user: OrlandoHugBot
paper_authors._id: 6822cf4c3b7392637edd756c
paper_authors.name: Jiangbo Pei
paper_authors._id: 6822cf4c3b7392637edd756d
paper_authors.name: Wei Shen
paper_authors._id: 6822cf4c3b7392637edd756e
paper_authors.name: Yi Peng
paper_authors._id: 6822cf4c3b7392637edd756f
paper_authors.name: Yunzhuo Hao
paper_authors._id: 6822cf4c3b7392637edd7570
paper_authors.name: Weijie Qiu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:56:35.421Z
paper_authors._id: 660aab2c878289c5b34f9e97
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg
paper_authors.fullname: weijie qiu
paper_authors.type: user
paper_authors.user: qiuwj
paper_authors._id: 6822cf4c3b7392637edd7571
paper_authors.name: Ai Jian
paper_authors._id: 6822cf4c3b7392637edd7572
paper_authors.name: Tianyidan Xie
paper_authors._id: 6822cf4c3b7392637edd7573
paper_authors.name: Xuchen Song
paper_authors._id: 6822cf4c3b7392637edd7574
paper_authors.name: Yang Liu
paper_authors._id: 6822cf4c3b7392637edd7575
paper_authors.name: Yahui Zhou
paper_title: Skywork-VL Reward: An Effective Reward Model for Multimodal
    Understanding and Reasoning
paper_summary: We propose Skywork-VL Reward, a multimodal reward model that provides reward
  signals for both multimodal understanding and reasoning tasks. Our technical
  approach comprises two key components: First, we construct a large-scale
  multimodal preference dataset that covers a wide range of tasks and scenarios,
  with responses collected from both standard vision-language models (VLMs) and
  advanced VLM reasoners. Second, we design a reward model architecture based on
  Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
  fine-tuning using pairwise ranking loss on pairwise preference data.
  Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
  results on multimodal VL-RewardBench and exhibits competitive performance on
  the text-only RewardBench benchmark. Furthermore, preference data constructed
  based on our Skywork-VL Reward proves highly effective for training Mixed
  Preference Optimization (MPO), leading to significant improvements in
  multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
  a significant advancement toward general-purpose, reliable reward models for
  multimodal alignment. Our model has been publicly released to promote
  transparency and reproducibility.
paper_ai_summary: Skywork-VL Reward is a multimodal reward model that uses a large-scale preference dataset and Qwen2.5-VL-7B-Instruct architecture to achieve state-of-the-art performance in multimodal reasoning.
paper_ai_keywords: multimodal reward model multimodal understanding Qwen2.5-VL-7B-Instruct reward head multi-stage fine-tuning pairwise ranking loss multimodal VL-RewardBench text-only RewardBench Mixed Preference Optimization multimodal reasoning capabilities general-purpose reward models multimodal alignment