title: OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming
    Video Contexts
summary: The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has
  propelled the development of Omni language models, designed to process and
  proactively respond to continuous streams of multi-modal data. Despite their
  potential, evaluating their real-world interactive capabilities in streaming
  video contexts remains a formidable challenge. In this work, we introduce
  OmniMMI, a comprehensive multi-modal interaction benchmark tailored for
  OmniLLMs in streaming video contexts. OmniMMI encompasses over 1,121 videos and
  2,290 questions, addressing two critical yet underexplored challenges in
  existing video benchmarks: streaming video understanding and proactive
  reasoning, across six distinct subtasks. Moreover, we propose a novel
  framework, Multi-modal Multiplexing Modeling (M4), designed to enable an
  inference-efficient streaming model that can see, listen while generating.
paper_authors._id: 67eb58e5969b278277adb831
paper_authors.name: Yuxuan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T09:42:25.151Z
paper_authors._id: 60b9e6837946aff342f734ae
paper_authors.avatarUrl: /avatars/a711a6aa35757dfd7b78b26098a964fc.svg
paper_authors.fullname: Yuxuan Wang
paper_authors.type: user
paper_authors.user: ColorfulAI
paper_authors._id: 67eb58e5969b278277adb832
paper_authors.name: Yueqian Wang
paper_authors._id: 67eb58e5969b278277adb833
paper_authors.name: Bo Chen
paper_authors._id: 67eb58e5969b278277adb834
paper_authors.name: Tong Wu
paper_authors._id: 67eb58e5969b278277adb835
paper_authors.name: Dongyan Zhao
paper_authors._id: 67eb58e5969b278277adb836
paper_authors.name: Zilong Zheng
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-04-01T03:09:26.871Z
paper_authors._id: 63a95a6a7930fa8c7dd63d4e
paper_authors.avatarUrl: /avatars/d9d0420f7ddfe2f3a7e029fb05f1c89f.svg
paper_authors.fullname: Zilong Zheng
paper_authors.type: user
paper_authors.user: zlzheng
paper_title: OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming
    Video Contexts
paper_summary: The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has
  propelled the development of Omni language models, designed to process and
  proactively respond to continuous streams of multi-modal data. Despite their
  potential, evaluating their real-world interactive capabilities in streaming
  video contexts remains a formidable challenge. In this work, we introduce
  OmniMMI, a comprehensive multi-modal interaction benchmark tailored for
  OmniLLMs in streaming video contexts. OmniMMI encompasses over 1,121 videos and
  2,290 questions, addressing two critical yet underexplored challenges in
  existing video benchmarks: streaming video understanding and proactive
  reasoning, across six distinct subtasks. Moreover, we propose a novel
  framework, Multi-modal Multiplexing Modeling (M4), designed to enable an
  inference-efficient streaming model that can see, listen while generating.
paper_ai_summary: OmniMMI is a benchmark and M4 is a framework aimed at evaluating and enabling inference-efficient multi-modal processing and proactive reasoning in streaming video contexts for OmniLLMs.
paper_ai_keywords: multi-modal language models Omni language models multi-modal interaction benchmark streaming video understanding proactive reasoning Multi-modal Multiplexing Modeling inference-efficient streaming model