InterFeedback: Unveiling Interactive Intelligence of Large Multimodal
  Models via Human Feedback
Existing benchmarks do not test Large Multimodal Models (LMMs) on their
interactive intelligence with human users which is vital for developing
general-purpose AI assistants. We design InterFeedback, an interactive
framework, which can be applied to any LMM and dataset to assess this ability
autonomously. On top of this, we introduce InterFeedback-Bench which evaluates
interactive intelligence using two representative datasets, MMMU-Pro and
MathVerse, to test 10 different open-source LMMs. Additionally, we present
InterFeedback-Human, a newly collected dataset of 120 cases designed for
manually testing interactive performance in leading models such as OpenAI-o1
and Claude-3.5-Sonnet. Our evaluation results show that even state-of-the-art
LMM (like OpenAI-o1) can correct their results through human feedback less than
50%. Our findings point to the need for methods that can enhance the LMMs'
capability to interpret and benefit from feedback.
67bbdcec79fcd85f09ddd869
Henry Hengyuan Zhao
admin_assigned
2025-02-24T13:32:25.051Z
647d7eb9770c299e56f5b39b
https://cdn-avatars.huggingface.co/v1/production/uploads/647d7eb9770c299e56f5b39b/CC5JJgkyLkXOxw-BeT4G5.jpeg
Henry Hengyuan Zhao
user
hhenryz
67bbdcec79fcd85f09ddd86a
Wenqi Pei
claimed_verified
2025-03-24T08:55:15.243Z
65e7d63b14856e8859f1924c
/avatars/bc58ab252c7b4d95ff99e4506fb8d3e9.svg
Pei Wenqi
user
CedPei
67bbdcec79fcd85f09ddd86b
Yifei Tao
admin_assigned
2025-02-24T15:52:28.951Z
6684c542cc72cbbde88ccf55
/avatars/dafa8b5b44dbb8be859fbae94a6cd953.svg
yifeitao
user
yifeitao
67bbdcec79fcd85f09ddd86c
Haiyang Mei
admin_assigned
2025-02-24T15:52:15.485Z
66fcfa6e05638227c44233a9
/avatars/4a88765c7f5c5ca77da6d21eb01f73e0.svg
Haiyang Mei
user
meihaiyang
67bbdcec79fcd85f09ddd86d
Mike Zheng Shou
admin_assigned
2025-02-24T15:52:21.365Z
661ab3da2b14565c7acccf5c
/avatars/fa4fc03664803e02aede4d4c3d50b393.svg
Mike Zheng Shou
user
AnalMom
InterFeedback: Unveiling Interactive Intelligence of Large Multimodal
  Models via Human Feedback
Existing benchmarks do not test Large Multimodal Models (LMMs) on their
interactive intelligence with human users which is vital for developing
general-purpose AI assistants. We design InterFeedback, an interactive
framework, which can be applied to any LMM and dataset to assess this ability
autonomously. On top of this, we introduce InterFeedback-Bench which evaluates
interactive intelligence using two representative datasets, MMMU-Pro and
MathVerse, to test 10 different open-source LMMs. Additionally, we present
InterFeedback-Human, a newly collected dataset of 120 cases designed for
manually testing interactive performance in leading models such as OpenAI-o1
and Claude-3.5-Sonnet. Our evaluation results show that even state-of-the-art
LMM (like OpenAI-o1) can correct their results through human feedback less than
50%. Our findings point to the need for methods that can enhance the LMMs'
capability to interpret and benefit from feedback.
InterFeedback evaluates the interactive intelligence of Large Multimodal Models (LMMs) using automated and manual benchmarks, revealing limitations in how these models process human feedback.
Large Multimodal Models LMMs interactive intelligence human feedback InterFeedback InterFeedback-Bench MMMU-Pro MathVerse InterFeedback-Human