WikiVideo: Article Generation from Multiple Videos
We present the challenging task of automatically creating a high-level
Wikipedia-style article that aggregates information from multiple diverse
videos about real-world events, such as natural disasters or political
elections. Videos are intuitive sources for retrieval-augmented generation
(RAG), but most contemporary RAG workflows focus heavily on text and existing
methods for video-based summarization focus on low-level scene understanding
rather than high-level event semantics. To close this gap, we introduce
WikiVideo, a benchmark consisting of expert-written articles and densely
annotated videos that provide evidence for articles' claims, facilitating the
integration of video into RAG pipelines and enabling the creation of in-depth
content that is grounded in multimodal sources. We further propose
Collaborative Article Generation (CAG), a novel interactive method for article
creation from multiple videos. CAG leverages an iterative interaction between
an r1-style reasoning model and a VideoLLM to draw higher level inferences
about the target event than is possible with VideoLLMs alone, which fixate on
low-level visual features. We benchmark state-of-the-art VideoLLMs and CAG in
both oracle retrieval and RAG settings and find that CAG consistently
outperforms alternative methods, while suggesting intriguing avenues for future
work.
67efffab5e8b84102499ebfc
Alexander Martin
claimed_verified
2025-04-08T06:53:55.294Z
644ffb154f731658826945c8
/avatars/46fa3a15c2b974692bee5b6ed6468be0.svg
Alex Martin
user
alexmartin1722
67efffab5e8b84102499ebfd
Reno Kriz
67efffab5e8b84102499ebfe
William Gantt Walden
67efffab5e8b84102499ebff
Kate Sanders
67efffab5e8b84102499ec00
Hannah Recknor
67efffab5e8b84102499ec01
Eugene Yang
67efffab5e8b84102499ec02
Francis Ferraro
67efffab5e8b84102499ec03
Benjamin Van Durme
WikiVideo: Article Generation from Multiple Videos
We present the challenging task of automatically creating a high-level
Wikipedia-style article that aggregates information from multiple diverse
videos about real-world events, such as natural disasters or political
elections. Videos are intuitive sources for retrieval-augmented generation
(RAG), but most contemporary RAG workflows focus heavily on text and existing
methods for video-based summarization focus on low-level scene understanding
rather than high-level event semantics. To close this gap, we introduce
WikiVideo, a benchmark consisting of expert-written articles and densely
annotated videos that provide evidence for articles' claims, facilitating the
integration of video into RAG pipelines and enabling the creation of in-depth
content that is grounded in multimodal sources. We further propose
Collaborative Article Generation (CAG), a novel interactive method for article
creation from multiple videos. CAG leverages an iterative interaction between
an r1-style reasoning model and a VideoLLM to draw higher level inferences
about the target event than is possible with VideoLLMs alone, which fixate on
low-level visual features. We benchmark state-of-the-art VideoLLMs and CAG in
both oracle retrieval and RAG settings and find that CAG consistently
outperforms alternative methods, while suggesting intriguing avenues for future
work.
WikiVideo proposes Collaborative Article Generation (CAG) to enhance high-level event summarization from videos by integrating r1-style reasoning and VideoLLM for better inferences compared to state-of-the-art VideoLLMs.
retrieval-augmented generation RAG WikiVideo Collaborative Article Generation CAG r1-style reasoning model VideoLLM high-level event semantics video-based summarization multimodal sources low-level visual features oracle retrieval