Scaling Up Video Summarization Pretraining with Large Language Models
<|reference_start|>Scaling Up Video Summarization Pretraining with Large Language Models: Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem. However, existing video summarization datasets are notably limited in their size, constraining the effectiveness of state-of-the-art methods for generalization. Our work aims to overcome this limitation by capitalizing on the abundance of long-form videos with dense speech-to-video alignment and the remarkable capabilities of recent large language models (LLMs) in summarizing long text. We introduce an automated and scalable pipeline for generating a large-scale video summarization dataset using LLMs as Oracle summarizers. By leveraging the generated dataset, we analyze the limitations of existing approaches and propose a new video summarization model that effectively addresses them. To facilitate further research in the field, our work also presents a new benchmark dataset that contains 1200 long videos each with high-quality summaries annotated by professionals. Extensive experiments clearly indicate that our proposed approach sets a new state-of-the-art in video summarization across several benchmarks.<|reference_end|>
@article{argaw2024scaling,
    title={Scaling Up Video Summarization Pretraining with Large Language Models},
    author={Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron, Hanieh
  Deilamsalehy, Trung Bui, Zhaowen Wang, Franck Dernoncourt, Joon Son Chung},
    journal={arXiv preprint arXiv:2404.03398},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.03398},
    primaryClass={cs.CV}
}