title: EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an
    Egocentric World?
summary: The emergence of multimodal large language models (MLLMs) has driven
  breakthroughs in egocentric vision applications. These applications necessitate
  persistent, context-aware understanding of objects, as users interact with
  tools in dynamic and cluttered environments. However, existing embodied
  benchmarks primarily focus on static scene exploration, emphasizing object's
  appearance and spatial attributes while neglecting the assessment of dynamic
  changes arising from users' interactions. To address this gap, we introduce
  EOC-Bench, an innovative benchmark designed to systematically evaluate
  object-centric embodied cognition in dynamic egocentric scenarios. Specially,
  EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three
  temporal categories: Past, Present, and Future, covering 11 fine-grained
  evaluation dimensions and 3 visual object referencing types. To ensure thorough
  assessment, we develop a mixed-format human-in-the-loop annotation framework
  with four types of questions and design a novel multi-scale temporal accuracy
  metric for open-ended temporal evaluation. Based on EOC-Bench, we conduct
  comprehensive evaluations of various proprietary, open-source, and object-level
  MLLMs. EOC-Bench serves as a crucial tool for advancing the embodied object
  cognitive capabilities of MLLMs, establishing a robust foundation for
  developing reliable core models for embodied systems.
paper_authors._id: 68425719ba04d3ceff5bea29
paper_authors.name: Yuqian Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-06T07:41:05.846Z
paper_authors._id: 64a3fe3dde901eb01df12398
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a3fe3dde901eb01df12398/Js2bEx4rxKuEKVt5z9I2D.jpeg
paper_authors.fullname: YuqianYuan
paper_authors.type: user
paper_authors.user: CircleRadon
paper_authors._id: 68425719ba04d3ceff5bea2a
paper_authors.name: Ronghao Dang
paper_authors._id: 68425719ba04d3ceff5bea2b
paper_authors.name: Long Li
paper_authors._id: 68425719ba04d3ceff5bea2c
paper_authors.name: Wentong Li
paper_authors._id: 68425719ba04d3ceff5bea2d
paper_authors.name: Dian Jiao
paper_authors._id: 68425719ba04d3ceff5bea2e
paper_authors.name: Xin Li
paper_authors._id: 68425719ba04d3ceff5bea2f
paper_authors.name: Deli Zhao
paper_authors._id: 68425719ba04d3ceff5bea30
paper_authors.name: Fan Wang
paper_authors._id: 68425719ba04d3ceff5bea31
paper_authors.name: Wenqiao Zhang
paper_authors._id: 68425719ba04d3ceff5bea32
paper_authors.name: Jun Xiao
paper_authors._id: 68425719ba04d3ceff5bea33
paper_authors.name: Yueting Zhuang
paper_title: EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an
    Egocentric World?
paper_summary: The emergence of multimodal large language models (MLLMs) has driven
  breakthroughs in egocentric vision applications. These applications necessitate
  persistent, context-aware understanding of objects, as users interact with
  tools in dynamic and cluttered environments. However, existing embodied
  benchmarks primarily focus on static scene exploration, emphasizing object's
  appearance and spatial attributes while neglecting the assessment of dynamic
  changes arising from users' interactions. To address this gap, we introduce
  EOC-Bench, an innovative benchmark designed to systematically evaluate
  object-centric embodied cognition in dynamic egocentric scenarios. Specially,
  EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three
  temporal categories: Past, Present, and Future, covering 11 fine-grained
  evaluation dimensions and 3 visual object referencing types. To ensure thorough
  assessment, we develop a mixed-format human-in-the-loop annotation framework
  with four types of questions and design a novel multi-scale temporal accuracy
  metric for open-ended temporal evaluation. Based on EOC-Bench, we conduct
  comprehensive evaluations of various proprietary, open-source, and object-level
  MLLMs. EOC-Bench serves as a crucial tool for advancing the embodied object
  cognitive capabilities of MLLMs, establishing a robust foundation for
  developing reliable core models for embodied systems.
paper_ai_summary: EOC-Bench introduces a benchmark to evaluate dynamic object-centric cognition in egocentric vision applications, focusing on temporal and interactive aspects not covered by existing benchmarks.
paper_ai_keywords: multimodal large language models (MLLMs) egocentric vision embodied benchmarks object-centric embodied cognition QA pairs temporal accuracy metric