title: LLM-AD: Large Language Model based Audio Description System
summary: The development of Audio Description (AD) has been a pivotal step forward in
  making video content more accessible and inclusive. Traditionally, AD
  production has demanded a considerable amount of skilled labor, while existing
  automated approaches still necessitate extensive training to integrate
  multimodal inputs and tailor the output from a captioning style to an AD style.
  In this paper, we introduce an automated AD generation pipeline that harnesses
  the potent multimodal and instruction-following capacities of GPT-4V(ision).
  Notably, our methodology employs readily available components, eliminating the
  need for additional training. It produces ADs that not only comply with
  established natural language AD production standards but also maintain
  contextually consistent character information across frames, courtesy of a
  tracking-based character recognition module. A thorough analysis on the MAD
  dataset reveals that our approach achieves a performance on par with
  learning-based methods in automated AD production, as substantiated by a CIDEr
  score of 20.5.
paper_authors._id: 663449d7fa10518a897fb08e
paper_authors.name: Peng Chu
paper_authors._id: 663449d7fa10518a897fb08f
paper_authors.name: Jiang Wang
paper_authors._id: 663449d7fa10518a897fb090
paper_authors.name: Andre Abrantes
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-03T08:01:37.280Z
paper_authors._id: 64b6add720b541022a13e9c4
paper_authors.avatarUrl: /avatars/d952f4a3f6f1205eda4d540c7bfd7d1e.svg
paper_authors.fullname: Andre Abrantes
paper_authors.type: user
paper_authors.user: abrantes
paper_title: LLM-AD: Large Language Model based Audio Description System
paper_summary: The development of Audio Description (AD) has been a pivotal step forward in
  making video content more accessible and inclusive. Traditionally, AD
  production has demanded a considerable amount of skilled labor, while existing
  automated approaches still necessitate extensive training to integrate
  multimodal inputs and tailor the output from a captioning style to an AD style.
  In this paper, we introduce an automated AD generation pipeline that harnesses
  the potent multimodal and instruction-following capacities of GPT-4V(ision).
  Notably, our methodology employs readily available components, eliminating the
  need for additional training. It produces ADs that not only comply with
  established natural language AD production standards but also maintain
  contextually consistent character information across frames, courtesy of a
  tracking-based character recognition module. A thorough analysis on the MAD
  dataset reveals that our approach achieves a performance on par with
  learning-based methods in automated AD production, as substantiated by a CIDEr
  score of 20.5.
paper_ai_summary: An automated pipeline using GPT-4V and tracking-based character recognition generates high-quality, contextually consistent audio descriptions without additional training.
paper_ai_keywords: GPT-4V multimodal instruction-following tracking-based character recognition MAD dataset CIDEr score