title: TEMPURA: Temporal Event Masked Prediction and Understanding for
    Reasoning in Action
summary: Understanding causal event relationships and achieving fine-grained temporal
  grounding in videos remain challenging for vision-language models. Existing
  methods either compress video tokens to reduce temporal resolution, or treat
  videos as unsegmented streams, which obscures fine-grained event boundaries and
  limits the modeling of causal dependencies. We propose TEMPURA (Temporal Event
  Masked Prediction and Understanding for Reasoning in Action), a two-stage
  training framework that enhances video temporal understanding. TEMPURA first
  applies masked event prediction reasoning to reconstruct missing events and
  generate step-by-step causal explanations from dense event annotations, drawing
  inspiration from effective infilling techniques. TEMPURA then learns to perform
  video segmentation and dense captioning to decompose videos into
  non-overlapping events with detailed, timestamp-aligned descriptions. We train
  TEMPURA on VER, a large-scale dataset curated by us that comprises 1M training
  instances and 500K videos with temporally aligned event descriptions and
  structured reasoning steps. Experiments on temporal grounding and highlight
  detection benchmarks demonstrate that TEMPURA outperforms strong baseline
  models, confirming that integrating causal reasoning with fine-grained temporal
  segmentation leads to improved video understanding.
paper_authors._id: 6819814653612b577df718e7
paper_authors.name: Jen-Hao Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:33:15.728Z
paper_authors._id: 65cd4d6256671dee8ee46392
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65cd4d6256671dee8ee46392/SH30XVQnGiYqYQIeDk3na.jpeg
paper_authors.fullname: Jen-Hao (Andy) Cheng
paper_authors.type: user
paper_authors.user: andaba
paper_authors._id: 6819814653612b577df718e8
paper_authors.name: Vivian Wang
paper_authors._id: 6819814653612b577df718e9
paper_authors.name: Huayu Wang
paper_authors._id: 6819814653612b577df718ea
paper_authors.name: Huapeng Zhou
paper_authors._id: 6819814653612b577df718eb
paper_authors.name: Yi-Hao Peng
paper_authors._id: 6819814653612b577df718ec
paper_authors.name: Hou-I Liu
paper_authors._id: 6819814653612b577df718ed
paper_authors.name: Hsiang-Wei Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-06T09:13:19.727Z
paper_authors._id: 647e4e8da49bffab5d72fbe0
paper_authors.avatarUrl: /avatars/c5fb00019c7cea23fe3351ecb1e43195.svg
paper_authors.fullname: Hsiang-Wei Huang
paper_authors.type: user
paper_authors.user: hsiangwei0903
paper_authors._id: 6819814653612b577df718ee
paper_authors.name: Kuang-Ming Chen
paper_authors._id: 6819814653612b577df718ef
paper_authors.name: Cheng-Yen Yang
paper_authors._id: 6819814653612b577df718f0
paper_authors.name: Wenhao Chai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-06T09:13:37.240Z
paper_authors._id: 637c7503fe115289cfecbe6b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676361945047-637c7503fe115289cfecbe6b.jpeg
paper_authors.fullname: Wenhao Chai
paper_authors.type: user
paper_authors.user: wchai
paper_authors._id: 6819814653612b577df718f1
paper_authors.name: Yi-Ling Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-06T09:13:43.313Z
paper_authors._id: 65f8cb651e0c65c13a2b906a
paper_authors.avatarUrl: /avatars/ffc8ac8f29ab1a3142fe5fab1b2302ca.svg
paper_authors.fullname: Yi-Ling Chen
paper_authors.type: user
paper_authors.user: yilche
paper_authors._id: 6819814653612b577df718f2
paper_authors.name: Vibhav Vineet
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-06T09:13:49.482Z
paper_authors._id: 63c8527becdb7c9fdd9cacc6
paper_authors.avatarUrl: /avatars/c8a3f5e1e5159ae5ead41bd9fc2b9b34.svg
paper_authors.fullname: Vibhav Vineet
paper_authors.type: user
paper_authors.user: vibhav-vineet
paper_authors._id: 6819814653612b577df718f3
paper_authors.name: Qin Cai
paper_authors._id: 6819814653612b577df718f4
paper_authors.name: Jenq-Neng Hwang
paper_title: TEMPURA: Temporal Event Masked Prediction and Understanding for
    Reasoning in Action
paper_summary: Understanding causal event relationships and achieving fine-grained temporal
  grounding in videos remain challenging for vision-language models. Existing
  methods either compress video tokens to reduce temporal resolution, or treat
  videos as unsegmented streams, which obscures fine-grained event boundaries and
  limits the modeling of causal dependencies. We propose TEMPURA (Temporal Event
  Masked Prediction and Understanding for Reasoning in Action), a two-stage
  training framework that enhances video temporal understanding. TEMPURA first
  applies masked event prediction reasoning to reconstruct missing events and
  generate step-by-step causal explanations from dense event annotations, drawing
  inspiration from effective infilling techniques. TEMPURA then learns to perform
  video segmentation and dense captioning to decompose videos into
  non-overlapping events with detailed, timestamp-aligned descriptions. We train
  TEMPURA on VER, a large-scale dataset curated by us that comprises 1M training
  instances and 500K videos with temporally aligned event descriptions and
  structured reasoning steps. Experiments on temporal grounding and highlight
  detection benchmarks demonstrate that TEMPURA outperforms strong baseline
  models, confirming that integrating causal reasoning with fine-grained temporal
  segmentation leads to improved video understanding.
paper_ai_summary: TEMPURA, a two-stage training framework, enhances video temporal understanding by reconstructing missing events, generating causal explanations, and performing dense video segmentation and captioning using a large-scale dataset.
paper_ai_keywords: TEMPURA masked event prediction causal explanations dense event annotations infilling techniques video segmentation dense captioning temporally aligned event descriptions structured reasoning steps temporal grounding highlight detection