title: Flex-Judge: Think Once, Judge Anywhere
summary: Human-generated reward signals are critical for aligning generative models
  with human preferences, guiding both training and inference-time evaluations.
  While large language models (LLMs) employed as proxy evaluators, i.e.,
  LLM-as-a-Judge, significantly reduce the costs associated with manual
  annotations, they typically require extensive modality-specific training data
  and fail to generalize well across diverse multimodal tasks. In this paper, we
  propose Flex-Judge, a reasoning-guided multimodal judge model that leverages
  minimal textual reasoning data to robustly generalize across multiple
  modalities and evaluation formats. Our core intuition is that structured
  textual reasoning explanations inherently encode generalizable decision-making
  patterns, enabling an effective transfer to multimodal judgments, e.g., with
  images or videos. Empirical results demonstrate that Flex-Judge, despite being
  trained on significantly fewer text data, achieves competitive or superior
  performance compared to state-of-the-art commercial APIs and extensively
  trained multimodal evaluators. Notably, Flex-Judge presents broad impact in
  modalities like molecule, where comprehensive evaluation benchmarks are scarce,
  underscoring its practical value in resource-constrained domains. Our framework
  highlights reasoning-based text supervision as a powerful, cost-effective
  alternative to traditional annotation-intensive approaches, substantially
  advancing scalable multimodal model-as-a-judge.
paper_authors._id: 6835268212dd354d6acdacbf
paper_authors.name: Jongwoo Ko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:46.843Z
paper_authors._id: 64b7628af902508f0d7ae112
paper_authors.avatarUrl: /avatars/83c155254486e80c1dfd14676fdf9215.svg
paper_authors.fullname: Jongwoo Ko
paper_authors.type: user
paper_authors.user: jongwooko
paper_authors._id: 6835268212dd354d6acdacc0
paper_authors.name: Sungnyun Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T08:15:10.751Z
paper_authors._id: 63f0c2ac9cf89c9ed1bdd25c
paper_authors.avatarUrl: /avatars/856b2cb482250fb83c6fe793e29dfd74.svg
paper_authors.fullname: Sungnyun Kim
paper_authors.type: user
paper_authors.user: sungnyun
paper_authors._id: 6835268212dd354d6acdacc1
paper_authors.name: Sungwoo Cho
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T12:53:28.976Z
paper_authors._id: 67516e5e36d9d33143027f5e
paper_authors.avatarUrl: /avatars/6a34f26a78b849476b30ddc765905bbb.svg
paper_authors.fullname: Sungwoo Cho
paper_authors.type: user
paper_authors.user: SeoulForest
paper_authors._id: 6835268212dd354d6acdacc2
paper_authors.name: Se-Young Yun
paper_title: Flex-Judge: Think Once, Judge Anywhere
paper_summary: Human-generated reward signals are critical for aligning generative models
  with human preferences, guiding both training and inference-time evaluations.
  While large language models (LLMs) employed as proxy evaluators, i.e.,
  LLM-as-a-Judge, significantly reduce the costs associated with manual
  annotations, they typically require extensive modality-specific training data
  and fail to generalize well across diverse multimodal tasks. In this paper, we
  propose Flex-Judge, a reasoning-guided multimodal judge model that leverages
  minimal textual reasoning data to robustly generalize across multiple
  modalities and evaluation formats. Our core intuition is that structured
  textual reasoning explanations inherently encode generalizable decision-making
  patterns, enabling an effective transfer to multimodal judgments, e.g., with
  images or videos. Empirical results demonstrate that Flex-Judge, despite being
  trained on significantly fewer text data, achieves competitive or superior
  performance compared to state-of-the-art commercial APIs and extensively
  trained multimodal evaluators. Notably, Flex-Judge presents broad impact in
  modalities like molecule, where comprehensive evaluation benchmarks are scarce,
  underscoring its practical value in resource-constrained domains. Our framework
  highlights reasoning-based text supervision as a powerful, cost-effective
  alternative to traditional annotation-intensive approaches, substantially
  advancing scalable multimodal model-as-a-judge.
paper_ai_summary: Flex-Judge uses minimal textual reasoning data to generalize across multiple modalities and evaluation formats, outperforming state-of-the-art models in multimodal evaluations.
paper_ai_keywords: reasoning-guided multimodal judge model structured textual reasoning explanations generalizable decision-making patterns multimodal judgments molecule evaluations reasoning-based text supervision scalable multimodal model-as-a-judge