Flex-Judge: Think Once, Judge Anywhere
Human-generated reward signals are critical for aligning generative models
with human preferences, guiding both training and inference-time evaluations.
While large language models (LLMs) employed as proxy evaluators, i.e.,
LLM-as-a-Judge, significantly reduce the costs associated with manual
annotations, they typically require extensive modality-specific training data
and fail to generalize well across diverse multimodal tasks. In this paper, we
propose Flex-Judge, a reasoning-guided multimodal judge model that leverages
minimal textual reasoning data to robustly generalize across multiple
modalities and evaluation formats. Our core intuition is that structured
textual reasoning explanations inherently encode generalizable decision-making
patterns, enabling an effective transfer to multimodal judgments, e.g., with
images or videos. Empirical results demonstrate that Flex-Judge, despite being
trained on significantly fewer text data, achieves competitive or superior
performance compared to state-of-the-art commercial APIs and extensively
trained multimodal evaluators. Notably, Flex-Judge presents broad impact in
modalities like molecule, where comprehensive evaluation benchmarks are scarce,
underscoring its practical value in resource-constrained domains. Our framework
highlights reasoning-based text supervision as a powerful, cost-effective
alternative to traditional annotation-intensive approaches, substantially
advancing scalable multimodal model-as-a-judge.
6835268212dd354d6acdacbf
Jongwoo Ko
claimed_verified
2025-05-28T09:00:46.843Z
64b7628af902508f0d7ae112
/avatars/83c155254486e80c1dfd14676fdf9215.svg
Jongwoo Ko
user
jongwooko
6835268212dd354d6acdacc0
Sungnyun Kim
claimed_verified
2025-05-27T08:15:10.751Z
63f0c2ac9cf89c9ed1bdd25c
/avatars/856b2cb482250fb83c6fe793e29dfd74.svg
Sungnyun Kim
user
sungnyun
6835268212dd354d6acdacc1
Sungwoo Cho
claimed_verified
2025-05-27T12:53:28.976Z
67516e5e36d9d33143027f5e
/avatars/6a34f26a78b849476b30ddc765905bbb.svg
Sungwoo Cho
user
SeoulForest
6835268212dd354d6acdacc2
Se-Young Yun
Flex-Judge: Think Once, Judge Anywhere
Human-generated reward signals are critical for aligning generative models
with human preferences, guiding both training and inference-time evaluations.
While large language models (LLMs) employed as proxy evaluators, i.e.,
LLM-as-a-Judge, significantly reduce the costs associated with manual
annotations, they typically require extensive modality-specific training data
and fail to generalize well across diverse multimodal tasks. In this paper, we
propose Flex-Judge, a reasoning-guided multimodal judge model that leverages
minimal textual reasoning data to robustly generalize across multiple
modalities and evaluation formats. Our core intuition is that structured
textual reasoning explanations inherently encode generalizable decision-making
patterns, enabling an effective transfer to multimodal judgments, e.g., with
images or videos. Empirical results demonstrate that Flex-Judge, despite being
trained on significantly fewer text data, achieves competitive or superior
performance compared to state-of-the-art commercial APIs and extensively
trained multimodal evaluators. Notably, Flex-Judge presents broad impact in
modalities like molecule, where comprehensive evaluation benchmarks are scarce,
underscoring its practical value in resource-constrained domains. Our framework
highlights reasoning-based text supervision as a powerful, cost-effective
alternative to traditional annotation-intensive approaches, substantially
advancing scalable multimodal model-as-a-judge.
Flex-Judge uses minimal textual reasoning data to generalize across multiple modalities and evaluation formats, outperforming state-of-the-art models in multimodal evaluations.
reasoning-guided multimodal judge model structured textual reasoning explanations generalizable decision-making patterns multimodal judgments molecule evaluations reasoning-based text supervision scalable multimodal model-as-a-judge