EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an
  Egocentric World?
The emergence of multimodal large language models (MLLMs) has driven
breakthroughs in egocentric vision applications. These applications necessitate
persistent, context-aware understanding of objects, as users interact with
tools in dynamic and cluttered environments. However, existing embodied
benchmarks primarily focus on static scene exploration, emphasizing object's
appearance and spatial attributes while neglecting the assessment of dynamic
changes arising from users' interactions. To address this gap, we introduce
EOC-Bench, an innovative benchmark designed to systematically evaluate
object-centric embodied cognition in dynamic egocentric scenarios. Specially,
EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three
temporal categories: Past, Present, and Future, covering 11 fine-grained
evaluation dimensions and 3 visual object referencing types. To ensure thorough
assessment, we develop a mixed-format human-in-the-loop annotation framework
with four types of questions and design a novel multi-scale temporal accuracy
metric for open-ended temporal evaluation. Based on EOC-Bench, we conduct
comprehensive evaluations of various proprietary, open-source, and object-level
MLLMs. EOC-Bench serves as a crucial tool for advancing the embodied object
cognitive capabilities of MLLMs, establishing a robust foundation for
developing reliable core models for embodied systems.
68425719ba04d3ceff5bea29
Yuqian Yuan
claimed_verified
2025-06-06T07:41:05.846Z
64a3fe3dde901eb01df12398
https://cdn-avatars.huggingface.co/v1/production/uploads/64a3fe3dde901eb01df12398/Js2bEx4rxKuEKVt5z9I2D.jpeg
YuqianYuan
user
CircleRadon
68425719ba04d3ceff5bea2a
Ronghao Dang
68425719ba04d3ceff5bea2b
Long Li
claimed_verified
2025-07-15T19:12:04.012Z
6365d83ce7a78348d82572b0
/avatars/d50587902cced2c3640fd5de82ff78dd.svg
ll
user
jianghuyihei
68425719ba04d3ceff5bea2c
Wentong Li
68425719ba04d3ceff5bea2d
Dian Jiao
68425719ba04d3ceff5bea2e
Xin Li
68425719ba04d3ceff5bea2f
Deli Zhao
68425719ba04d3ceff5bea30
Fan Wang
68425719ba04d3ceff5bea31
Wenqiao Zhang
68425719ba04d3ceff5bea32
Jun Xiao
68425719ba04d3ceff5bea33
Yueting Zhuang
EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an
  Egocentric World?
The emergence of multimodal large language models (MLLMs) has driven
breakthroughs in egocentric vision applications. These applications necessitate
persistent, context-aware understanding of objects, as users interact with
tools in dynamic and cluttered environments. However, existing embodied
benchmarks primarily focus on static scene exploration, emphasizing object's
appearance and spatial attributes while neglecting the assessment of dynamic
changes arising from users' interactions. To address this gap, we introduce
EOC-Bench, an innovative benchmark designed to systematically evaluate
object-centric embodied cognition in dynamic egocentric scenarios. Specially,
EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three
temporal categories: Past, Present, and Future, covering 11 fine-grained
evaluation dimensions and 3 visual object referencing types. To ensure thorough
assessment, we develop a mixed-format human-in-the-loop annotation framework
with four types of questions and design a novel multi-scale temporal accuracy
metric for open-ended temporal evaluation. Based on EOC-Bench, we conduct
comprehensive evaluations of various proprietary, open-source, and object-level
MLLMs. EOC-Bench serves as a crucial tool for advancing the embodied object
cognitive capabilities of MLLMs, establishing a robust foundation for
developing reliable core models for embodied systems.
EOC-Bench introduces a benchmark to evaluate dynamic object-centric cognition in egocentric vision applications, focusing on temporal and interactive aspects not covered by existing benchmarks.
multimodal large language models (MLLMs) egocentric vision embodied benchmarks object-centric embodied cognition QA pairs temporal accuracy metric