title: Temporal Preference Optimization for Long-Form Video Understanding
summary: Despite significant advancements in video large multimodal models
  (video-LMMs), achieving effective temporal grounding in long-form videos
  remains a challenge for existing models. To address this limitation, we propose
  Temporal Preference Optimization (TPO), a novel post-training framework
  designed to enhance the temporal grounding capabilities of video-LMMs through
  preference learning. TPO adopts a self-training approach that enables models to
  differentiate between well-grounded and less accurate temporal responses by
  leveraging curated preference datasets at two granularities: localized temporal
  grounding, which focuses on specific video segments, and comprehensive temporal
  grounding, which captures extended temporal dependencies across entire video
  sequences. By optimizing on these preference datasets, TPO significantly
  enhances temporal understanding while reducing reliance on manually annotated
  data. Extensive experiments on three long-form video understanding
  benchmarks--LongVideoBench, MLVU, and Video-MME--demonstrate the effectiveness
  of TPO across two state-of-the-art video-LMMs. Notably, LLaVA-Video-TPO
  establishes itself as the leading 7B model on the Video-MME benchmark,
  underscoring the potential of TPO as a scalable and efficient solution for
  advancing temporal reasoning in long-form video understanding. Project page:
  https://ruili33.github.io/tpo_website.
paper_authors._id: 679317f9d3ef2f790a539a28
paper_authors.name: Rui Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T10:17:34.282Z
paper_authors._id: 6785fc7d17a2dfa3720ec082
paper_authors.avatarUrl: /avatars/73e9d715bb16f14240c733c4843dfc22.svg
paper_authors.fullname: Rui Li
paper_authors.type: user
paper_authors.user: ruili0
paper_authors._id: 679317f9d3ef2f790a539a29
paper_authors.name: Xiaohan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:11:30.999Z
paper_authors._id: 65703fab7f50602340d23704
paper_authors.avatarUrl: /avatars/324c45f5fba9cd8c38a89b30427c06b4.svg
paper_authors.fullname: Xiaohan Wang
paper_authors.type: user
paper_authors.user: nicholswang
paper_authors._id: 679317f9d3ef2f790a539a2a
paper_authors.name: Yuhui Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:11:46.689Z
paper_authors._id: 62da55164398e21bf7f0e292
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
paper_authors.fullname: Yuhui Zhang
paper_authors.type: user
paper_authors.user: yuhuizhang
paper_authors._id: 679317f9d3ef2f790a539a2b
paper_authors.name: Zeyu Wang
paper_authors._id: 679317f9d3ef2f790a539a2c
paper_authors.name: Serena Yeung-Levy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:09:52.788Z
paper_authors._id: 677c8b2e92550a07fcad0f50
paper_authors.avatarUrl: /avatars/2be26e8f25e98cfe5b1d227ee0409cd0.svg
paper_authors.fullname: Serena Yeung-Levy
paper_authors.type: user
paper_authors.user: yeunglevy
paper_title: Temporal Preference Optimization for Long-Form Video Understanding
paper_summary: Despite significant advancements in video large multimodal models
  (video-LMMs), achieving effective temporal grounding in long-form videos
  remains a challenge for existing models. To address this limitation, we propose
  Temporal Preference Optimization (TPO), a novel post-training framework
  designed to enhance the temporal grounding capabilities of video-LMMs through
  preference learning. TPO adopts a self-training approach that enables models to
  differentiate between well-grounded and less accurate temporal responses by
  leveraging curated preference datasets at two granularities: localized temporal
  grounding, which focuses on specific video segments, and comprehensive temporal
  grounding, which captures extended temporal dependencies across entire video
  sequences. By optimizing on these preference datasets, TPO significantly
  enhances temporal understanding while reducing reliance on manually annotated
  data. Extensive experiments on three long-form video understanding
  benchmarks--LongVideoBench, MLVU, and Video-MME--demonstrate the effectiveness
  of TPO across two state-of-the-art video-LMMs. Notably, LLaVA-Video-TPO
  establishes itself as the leading 7B model on the Video-MME benchmark,
  underscoring the potential of TPO as a scalable and efficient solution for
  advancing temporal reasoning in long-form video understanding. Project page:
  https://ruili33.github.io/tpo_website.
paper_ai_keywords: Temporal Feature Optimization (TPO) temporal grounding temporal response preference learning self-training localized temporal grounding comprehensive temporal grounding temporal dependencies preference datasets temporal understanding manual annotated data long-form video understanding LongVideoBench MLVU Video-MME state-of-the-art video-LMMs LLaVA-Video-TPO