title: VideoAuteur: Towards Long Narrative Video Generation
summary: Recent video generation models have shown promising results in producing
  high-quality video clips lasting several seconds. However, these models face
  challenges in generating long sequences that convey clear and informative
  events, limiting their ability to support coherent narrations. In this paper,
  we present a large-scale cooking video dataset designed to advance long-form
  narrative generation in the cooking domain. We validate the quality of our
  proposed dataset in terms of visual fidelity and textual caption accuracy using
  state-of-the-art Vision-Language Models (VLMs) and video generation models,
  respectively. We further introduce a Long Narrative Video Director to enhance
  both visual and semantic coherence in generated videos and emphasize the role
  of aligning visual embeddings to achieve improved overall video quality. Our
  method demonstrates substantial improvements in generating visually detailed
  and semantically aligned keyframes, supported by finetuning techniques that
  integrate text and image embeddings within the video generation process.
  Project page: https://videoauteur.github.io/
paper_authors._id: 678604c71f54791d52bebe30
paper_authors.name: Junfei Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-14T08:28:41.422Z
paper_authors._id: 64b5ba6060274cbb296d6288
paper_authors.avatarUrl: /avatars/67e0343954dda6e92ed3f6e7976f9f87.svg
paper_authors.fullname: Junfei Xiao
paper_authors.type: user
paper_authors.user: lambertxiao
paper_authors._id: 678604c71f54791d52bebe31
paper_authors.name: Feng Cheng
paper_authors._id: 678604c71f54791d52bebe32
paper_authors.name: Lu Qi
paper_authors._id: 678604c71f54791d52bebe33
paper_authors.name: Liangke Gui
paper_authors._id: 678604c71f54791d52bebe34
paper_authors.name: Jiepeng Cen
paper_authors._id: 678604c71f54791d52bebe35
paper_authors.name: Zhibei Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-16T21:55:51.675Z
paper_authors._id: 645d5a44680734460f9e7742
paper_authors.avatarUrl: /avatars/66687e2c413acce55436d967071f8786.svg
paper_authors.fullname: Zhibei Ma
paper_authors.type: user
paper_authors.user: Brightmzb
paper_authors._id: 678604c71f54791d52bebe36
paper_authors.name: Alan Yuille
paper_authors._id: 678604c71f54791d52bebe37
paper_authors.name: Lu Jiang
paper_title: VideoAuteur: Towards Long Narrative Video Generation
paper_summary: Recent video generation models have shown promising results in producing
  high-quality video clips lasting several seconds. However, these models face
  challenges in generating long sequences that convey clear and informative
  events, limiting their ability to support coherent narrations. In this paper,
  we present a large-scale cooking video dataset designed to advance long-form
  narrative generation in the cooking domain. We validate the quality of our
  proposed dataset in terms of visual fidelity and textual caption accuracy using
  state-of-the-art Vision-Language Models (VLMs) and video generation models,
  respectively. We further introduce a Long Narrative Video Director to enhance
  both visual and semantic coherence in generated videos and emphasize the role
  of aligning visual embeddings to achieve improved overall video quality. Our
  method demonstrates substantial improvements in generating visually detailed
  and semantically aligned keyframes, supported by finetuning techniques that
  integrate text and image embeddings within the video generation process.
  Project page: https://videoauteur.github.io/
paper_ai_keywords: large-scale cooking video dataset long-form narrative generation Vision-Language Models (VLMs) Long Narrative Video Director visual embeddings semantic coherence generated videos finetuning techniques text and image embeddings video generation process visually detailed semantically aligned keyframes