title: Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
summary: We address the task of video chaptering, i.e., partitioning a long video
  timeline into semantic units and generating corresponding chapter titles. While
  relatively underexplored, automatic chaptering has the potential to enable
  efficient navigation and content retrieval in long-form videos. In this paper,
  we achieve strong chaptering performance on hour-long videos by efficiently
  addressing the problem in the text domain with our 'Chapter-Llama' framework.
  Specifically, we leverage a pretrained large language model (LLM) with large
  context window, and feed as input (i) speech transcripts and (ii) captions
  describing video frames, along with their respective timestamps. Given the
  inefficiency of exhaustively captioning all frames, we propose a lightweight
  speech-guided frame selection strategy based on speech transcript content, and
  experimentally demonstrate remarkable advantages. We train the LLM to output
  timestamps for the chapter boundaries, as well as free-form chapter titles.
  This simple yet powerful approach scales to processing one-hour long videos in
  a single forward pass. Our results demonstrate substantial improvements (e.g.,
  45.3 vs 26.7 F1 score) over the state of the art on the recent VidChapters-7M
  benchmark. To promote further research, we release our code and models at our
  project page.
paper_authors._id: 67eccbca1006da75eca94d24
paper_authors.name: Lucas Ventura
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-06T08:07:08.372Z
paper_authors._id: 62d1583d1e36881a57f041ab
paper_authors.avatarUrl: /avatars/33315f8431196e1f6c57162fa37f4eef.svg
paper_authors.fullname: Lucas
paper_authors.type: user
paper_authors.user: lucas-ventura
paper_authors._id: 67eccbca1006da75eca94d25
paper_authors.name: Antoine Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:28:15.216Z
paper_authors._id: 6512da122f79557a90b1717f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6512da122f79557a90b1717f/-rL2Iu_HHPEi1zWOL4cZi.jpeg
paper_authors.fullname: Antoine Yang
paper_authors.type: user
paper_authors.user: antoineyang
paper_authors._id: 67eccbca1006da75eca94d26
paper_authors.name: Cordelia Schmid
paper_authors._id: 67eccbca1006da75eca94d27
paper_authors.name: Gül Varol
paper_title: Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
paper_summary: We address the task of video chaptering, i.e., partitioning a long video
  timeline into semantic units and generating corresponding chapter titles. While
  relatively underexplored, automatic chaptering has the potential to enable
  efficient navigation and content retrieval in long-form videos. In this paper,
  we achieve strong chaptering performance on hour-long videos by efficiently
  addressing the problem in the text domain with our 'Chapter-Llama' framework.
  Specifically, we leverage a pretrained large language model (LLM) with large
  context window, and feed as input (i) speech transcripts and (ii) captions
  describing video frames, along with their respective timestamps. Given the
  inefficiency of exhaustively captioning all frames, we propose a lightweight
  speech-guided frame selection strategy based on speech transcript content, and
  experimentally demonstrate remarkable advantages. We train the LLM to output
  timestamps for the chapter boundaries, as well as free-form chapter titles.
  This simple yet powerful approach scales to processing one-hour long videos in
  a single forward pass. Our results demonstrate substantial improvements (e.g.,
  45.3 vs 26.7 F1 score) over the state of the art on the recent VidChapters-7M
  benchmark. To promote further research, we release our code and models at our
  project page.
paper_ai_summary: A framework using a pretrained large language model for video chaptering enhances performance by efficiently processing speech transcripts and selected video frames to generate chapter titles and timestamps.
paper_ai_keywords: pretrained large language model (LLM) large context window speech transcripts speech-guided frame selection VidChapters-7M benchmark