title: FocusedAD: Character-centric Movie Audio Description
summary: Movie Audio Description (AD) aims to narrate visual content during
  dialogue-free segments, particularly benefiting blind and visually impaired
  (BVI) audiences. Compared with general video captioning, AD demands
  plot-relevant narration with explicit character name references, posing unique
  challenges in movie understanding.To identify active main characters and focus
  on storyline-relevant regions, we propose FocusedAD, a novel framework that
  delivers character-centric movie audio descriptions. It includes: (i) a
  Character Perception Module(CPM) for tracking character regions and linking
  them to names; (ii) a Dynamic Prior Module(DPM) that injects contextual cues
  from prior ADs and subtitles via learnable soft prompts; and (iii) a Focused
  Caption Module(FCM) that generates narrations enriched with plot-relevant
  details and named characters. To overcome limitations in character
  identification, we also introduce an automated pipeline for building character
  query banks. FocusedAD achieves state-of-the-art performance on multiple
  benchmarks, including strong zero-shot results on MAD-eval-Named and our newly
  proposed Cinepile-AD dataset. Code and data will be released at
  https://github.com/Thorin215/FocusedAD .
paper_authors._id: 6801d4446d2188af01a9b6b6
paper_authors.name: Xiaojun Ye
paper_authors._id: 6801d4446d2188af01a9b6b7
paper_authors.name: Chun Wang
paper_authors._id: 6801d4446d2188af01a9b6b8
paper_authors.name: Yiren Song
paper_authors._id: 6801d4446d2188af01a9b6b9
paper_authors.name: Sheng Zhou
paper_authors._id: 6801d4446d2188af01a9b6ba
paper_authors.name: Liangcheng Li
paper_authors._id: 6801d4446d2188af01a9b6bb
paper_authors.name: Jiajun Bu
paper_title: FocusedAD: Character-centric Movie Audio Description
paper_summary: Movie Audio Description (AD) aims to narrate visual content during
  dialogue-free segments, particularly benefiting blind and visually impaired
  (BVI) audiences. Compared with general video captioning, AD demands
  plot-relevant narration with explicit character name references, posing unique
  challenges in movie understanding.To identify active main characters and focus
  on storyline-relevant regions, we propose FocusedAD, a novel framework that
  delivers character-centric movie audio descriptions. It includes: (i) a
  Character Perception Module(CPM) for tracking character regions and linking
  them to names; (ii) a Dynamic Prior Module(DPM) that injects contextual cues
  from prior ADs and subtitles via learnable soft prompts; and (iii) a Focused
  Caption Module(FCM) that generates narrations enriched with plot-relevant
  details and named characters. To overcome limitations in character
  identification, we also introduce an automated pipeline for building character
  query banks. FocusedAD achieves state-of-the-art performance on multiple
  benchmarks, including strong zero-shot results on MAD-eval-Named and our newly
  proposed Cinepile-AD dataset. Code and data will be released at
  https://github.com/Thorin215/FocusedAD .
paper_ai_summary: FocusedAD, a novel framework for movie audio descriptions, tracks characters, incorporates contextual cues, and generates plot-relevant narrations, achieving top performance on benchmarks and zero-shot results.
paper_ai_keywords: Character Perception Module Dynamic Prior Module Focused Caption Module character identification character query banks MAD-eval-Named Cinepile-AD dataset