title: Cockatiel: Ensembling Synthetic and Human Preferenced Training for
    Detailed Video Caption
summary: Video Detailed Captioning (VDC) is a crucial task for vision-language
  bridging, enabling fine-grained descriptions of complex video content. In this
  paper, we first comprehensively benchmark current state-of-the-art approaches
  and systematically identified two critical limitations: biased capability
  towards specific captioning aspect and misalignment with human preferences. To
  address these deficiencies, we propose Cockatiel, a novel three-stage training
  pipeline that ensembles synthetic and human-aligned training for improving VDC
  performance. In the first stage, we derive a scorer from a meticulously
  annotated dataset to select synthetic captions high-performing on certain
  fine-grained video-caption alignment and human-preferred while disregarding
  others. Then, we train Cockatiel-13B, using this curated dataset to infuse it
  with assembled model strengths and human preferences. Finally, we further
  distill Cockatiel-8B from Cockatiel-13B for the ease of usage. Extensive
  quantitative and qualitative experiments reflect the effectiveness of our
  method, as we not only set new state-of-the-art performance on VDCSCORE in a
  dimension-balanced way but also surpass leading alternatives on human
  preference by a large margin as depicted by the human evaluation results.
paper_authors._id: 67d2bd340860f2d7ff10e3dc
paper_authors.name: Luozheng Qin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:56:11.189Z
paper_authors._id: 66a9b3533d417b0baa9220a6
paper_authors.avatarUrl: /avatars/adc372bd24df1d3bf43258833411e8af.svg
paper_authors.fullname: Luozheng Qin
paper_authors.type: user
paper_authors.user: Fr0zencr4nE
paper_authors._id: 67d2bd340860f2d7ff10e3dd
paper_authors.name: Zhiyu Tan
paper_authors._id: 67d2bd340860f2d7ff10e3de
paper_authors.name: Mengping Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:56:25.200Z
paper_authors._id: 6304d630dae2eb7d084148c7
paper_authors.avatarUrl: /avatars/7d7a6ca99334bdae3ed1752ff40a8d94.svg
paper_authors.fullname: mengping yang
paper_authors.type: user
paper_authors.user: Kobeshegu
paper_authors._id: 67d2bd340860f2d7ff10e3df
paper_authors.name: Xiaomeng Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:56:33.077Z
paper_authors._id: 658ea92268d0b7633176b4ed
paper_authors.avatarUrl: /avatars/40173c9126dccfe78bc46b12c6ced8c8.svg
paper_authors.fullname: xiaomeng yang
paper_authors.type: user
paper_authors.user: xiaomengyang
paper_authors._id: 67d2bd340860f2d7ff10e3e0
paper_authors.name: Hao Li
paper_title: Cockatiel: Ensembling Synthetic and Human Preferenced Training for
    Detailed Video Caption
paper_summary: Video Detailed Captioning (VDC) is a crucial task for vision-language
  bridging, enabling fine-grained descriptions of complex video content. In this
  paper, we first comprehensively benchmark current state-of-the-art approaches
  and systematically identified two critical limitations: biased capability
  towards specific captioning aspect and misalignment with human preferences. To
  address these deficiencies, we propose Cockatiel, a novel three-stage training
  pipeline that ensembles synthetic and human-aligned training for improving VDC
  performance. In the first stage, we derive a scorer from a meticulously
  annotated dataset to select synthetic captions high-performing on certain
  fine-grained video-caption alignment and human-preferred while disregarding
  others. Then, we train Cockatiel-13B, using this curated dataset to infuse it
  with assembled model strengths and human preferences. Finally, we further
  distill Cockatiel-8B from Cockatiel-13B for the ease of usage. Extensive
  quantitative and qualitative experiments reflect the effectiveness of our
  method, as we not only set new state-of-the-art performance on VDCSCORE in a
  dimension-balanced way but also surpass leading alternatives on human
  preference by a large margin as depicted by the human evaluation results.
paper_ai_keywords: Cockatiel three-stage training pipeline synthetic and human-aligned training fine-grained video-caption alignment scorer meticulously annotated dataset curated dataset assembled model strengths human preferences VDCSCORE dimension-balanced way human evaluation results