title: Towards Retrieval Augmented Generation over Large Video Libraries
summary: Video content creators need efficient tools to repurpose content, a task that
  often requires complex manual or automated searches. Crafting a new video from
  large video libraries remains a challenge. In this paper we introduce the task
  of Video Library Question Answering (VLQA) through an interoperable
  architecture that applies Retrieval Augmented Generation (RAG) to video
  libraries. We propose a system that uses large language models (LLMs) to
  generate search queries, retrieving relevant video moments indexed by speech
  and visual metadata. An answer generation module then integrates user queries
  with this metadata to produce responses with specific video timestamps. This
  approach shows promise in multimedia content retrieval, and AI-assisted video
  content creation.
paper_authors._id: 667920938c58c5a12a9ca8c9
paper_authors.name: Yannis Tevissen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T07:32:54.486Z
paper_authors._id: 62cd4fc3299c0c2e0e42f249
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62cd4fc3299c0c2e0e42f249/4_frS9S9RXVY4fidEoJSZ.jpeg
paper_authors.fullname: Yannis Tevissen
paper_authors.type: user
paper_authors.user: YannisTevissen
paper_authors._id: 667920938c58c5a12a9ca8ca
paper_authors.name: Khalil Guetari
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T13:10:29.694Z
paper_authors._id: 64f70494b23f1414f73b1293
paper_authors.avatarUrl: /avatars/86c0bc7365c9cbcf09974f492ebc6fe6.svg
paper_authors.fullname: Khalil Guetari
paper_authors.type: user
paper_authors.user: KhalilGuetari
paper_authors._id: 667920938c58c5a12a9ca8cb
paper_authors.name: Frédéric Petitpont
paper_title: Towards Retrieval Augmented Generation over Large Video Libraries
paper_summary: Video content creators need efficient tools to repurpose content, a task that
  often requires complex manual or automated searches. Crafting a new video from
  large video libraries remains a challenge. In this paper we introduce the task
  of Video Library Question Answering (VLQA) through an interoperable
  architecture that applies Retrieval Augmented Generation (RAG) to video
  libraries. We propose a system that uses large language models (LLMs) to
  generate search queries, retrieving relevant video moments indexed by speech
  and visual metadata. An answer generation module then integrates user queries
  with this metadata to produce responses with specific video timestamps. This
  approach shows promise in multimedia content retrieval, and AI-assisted video
  content creation.
paper_ai_summary: The proposed system uses Retrieval Augmented Generation (RAG) and large language models to efficiently answer questions about video libraries by generating relevant search queries, retrieving video moments, and producing timestamped responses.
paper_ai_keywords: Retrieval Augmented Generation (RAG) large language models (LLMs) multimedia content retrieval AI-assisted video content creation