title: VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference
    Optimization for Large Video Models
summary: Large Video Models (LVMs) built upon Large Language Models (LLMs) have shown
  promise in video understanding but often suffer from misalignment with human
  intuition and video hallucination issues. To address these challenges, we
  introduce VistaDPO, a novel framework for Video Hierarchical Spatial-Temporal
  Direct Preference Optimization. VistaDPO enhances text-video preference
  alignment across three hierarchical levels: i) Instance Level, aligning overall
  video content with responses; ii) Temporal Level, aligning video temporal
  semantics with event descriptions; and iii) Perceptive Level, aligning spatial
  objects with language tokens. Given the lack of datasets for fine-grained
  video-language preference alignment, we construct VistaDPO-7k, a dataset of
  7.2K QA pairs annotated with chosen and rejected responses, along with
  spatial-temporal grounding information such as timestamps, keyframes, and
  bounding boxes. Extensive experiments on benchmarks such as Video
  Hallucination, Video QA, and Captioning performance tasks demonstrate that
  VistaDPO significantly improves the performance of existing LVMs, effectively
  mitigating video-language misalignment and hallucination. The code and data are
  available at https://github.com/HaroldChen19/VistaDPO.
paper_authors._id: 6801b62608d748addc187952
paper_authors.name: Haojian Huang
paper_authors._id: 6801b62608d748addc187953
paper_authors.name: Haodong Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:39:11.374Z
paper_authors._id: 6570450a78d7aca0c361a177
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/z0GrnXEsjK2_G-hFfQhKv.jpeg
paper_authors.fullname: Harold Chen
paper_authors.type: user
paper_authors.user: Harold328
paper_authors._id: 6801b62608d748addc187954
paper_authors.name: Shengqiong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:39:08.857Z
paper_authors._id: 64c139d867eff857ea51caa8
paper_authors.avatarUrl: /avatars/4b7b3f41c2e2cfa21dd43bbac6e081ae.svg
paper_authors.fullname: Shengqiong Wu
paper_authors.type: user
paper_authors.user: ChocoWu
paper_authors._id: 6801b62608d748addc187955
paper_authors.name: Meng Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-22T10:13:20.029Z
paper_authors._id: 64ad1c0bad6218d51a07b54e
paper_authors.avatarUrl: /avatars/0f84d9a51c6ca9bcef44de2d7c707d9b.svg
paper_authors.fullname: LUO MENG
paper_authors.type: user
paper_authors.user: Eureka-Leo
paper_authors._id: 6801b62608d748addc187956
paper_authors.name: Jinlan Fu
paper_authors._id: 6801b62608d748addc187957
paper_authors.name: Xinya Du
paper_authors._id: 6801b62608d748addc187958
paper_authors.name: Hanwang Zhang
paper_authors._id: 6801b62608d748addc187959
paper_authors.name: Hao Fei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:39:06.521Z
paper_authors._id: 647773a1168cb428e00e9a8f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647773a1168cb428e00e9a8f/NiRR3ScY6Plzjibfwy1hC.jpeg
paper_authors.fullname: Hao Fei
paper_authors.type: user
paper_authors.user: scofield7419
paper_title: VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference
    Optimization for Large Video Models
paper_summary: Large Video Models (LVMs) built upon Large Language Models (LLMs) have shown
  promise in video understanding but often suffer from misalignment with human
  intuition and video hallucination issues. To address these challenges, we
  introduce VistaDPO, a novel framework for Video Hierarchical Spatial-Temporal
  Direct Preference Optimization. VistaDPO enhances text-video preference
  alignment across three hierarchical levels: i) Instance Level, aligning overall
  video content with responses; ii) Temporal Level, aligning video temporal
  semantics with event descriptions; and iii) Perceptive Level, aligning spatial
  objects with language tokens. Given the lack of datasets for fine-grained
  video-language preference alignment, we construct VistaDPO-7k, a dataset of
  7.2K QA pairs annotated with chosen and rejected responses, along with
  spatial-temporal grounding information such as timestamps, keyframes, and
  bounding boxes. Extensive experiments on benchmarks such as Video
  Hallucination, Video QA, and Captioning performance tasks demonstrate that
  VistaDPO significantly improves the performance of existing LVMs, effectively
  mitigating video-language misalignment and hallucination. The code and data are
  available at https://github.com/HaroldChen19/VistaDPO.
paper_ai_summary: VistaDPO, a new framework, enhances text-video preference alignment through hierarchical spatial-temporal optimization, addressing misalignment and hallucination in large video models.
paper_ai_keywords: Large Video Models Large Language Models Video Hierarchical Spatial-Temporal Direct Preference Optimization Instance Level Temporal Level Perceptive Level VistaDPO-7k video hallucination Video QA Captioning