title: MovieLLM: Enhancing Long Video Understanding with AI-Generated Movies
summary: The development of multimodal models has marked a significant step forward in
  how machines understand videos. These models have shown promise in analyzing
  short video clips. However, when it comes to longer formats like movies, they
  often fall short. The main hurdles are the lack of high-quality, diverse video
  data and the intensive work required to collect or annotate such data. In the
  face of these challenges, we propose MovieLLM, a novel framework designed to
  create synthetic, high-quality data for long videos. This framework leverages
  the power of GPT-4 and text-to-image models to generate detailed scripts and
  corresponding visuals. Our approach stands out for its flexibility and
  scalability, making it a superior alternative to traditional data collection
  methods. Our extensive experiments validate that the data produced by MovieLLM
  significantly improves the performance of multimodal models in understanding
  complex video narratives, overcoming the limitations of existing datasets
  regarding scarcity and bias.
paper_authors._id: 65e7482deea9f3f4c447113a
paper_authors.name: Zhende Song
paper_authors._id: 65e7482deea9f3f4c447113b
paper_authors.name: Chenchen Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:41:19.684Z
paper_authors._id: 6455c0cebda0fbba412daa16
paper_authors.avatarUrl: /avatars/2963ea08c3e537273e03bd771fb4d279.svg
paper_authors.fullname: chenchen
paper_authors.type: user
paper_authors.user: wangchenchen
paper_authors._id: 65e7482deea9f3f4c447113c
paper_authors.name: Jiamu Sheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:41:07.264Z
paper_authors._id: 64a3dd87e2fdaf1897f8c88c
paper_authors.avatarUrl: /avatars/47e988bbfc8abbb580f12ae3020584b9.svg
paper_authors.fullname: Jiamu Sheng
paper_authors.type: user
paper_authors.user: Yooaena
paper_authors._id: 65e7482deea9f3f4c447113d
paper_authors.name: Chi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-08T07:43:06.501Z
paper_authors._id: 64196320ed725fef64419c2a
paper_authors.avatarUrl: /avatars/96feb22fb5e8931d6c9e0ea06148266f.svg
paper_authors.fullname: Chi Zhang
paper_authors.type: user
paper_authors.user: DrChiZhang
paper_authors._id: 65e7482deea9f3f4c447113e
paper_authors.name: Gang Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-07T09:15:31.796Z
paper_authors._id: 63417332c5565a4b8d43a0d8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63417332c5565a4b8d43a0d8/MmnYG7Wu2Z_lqCBvTfJmy.png
paper_authors.fullname: Gang Yu
paper_authors.type: user
paper_authors.user: skicy
paper_authors._id: 65e7482deea9f3f4c447113f
paper_authors.name: Jiayuan Fan
paper_authors._id: 65e7482deea9f3f4c4471140
paper_authors.name: Tao Chen
paper_title: MovieLLM: Enhancing Long Video Understanding with AI-Generated Movies
paper_summary: The development of multimodal models has marked a significant step forward in
  how machines understand videos. These models have shown promise in analyzing
  short video clips. However, when it comes to longer formats like movies, they
  often fall short. The main hurdles are the lack of high-quality, diverse video
  data and the intensive work required to collect or annotate such data. In the
  face of these challenges, we propose MovieLLM, a novel framework designed to
  create synthetic, high-quality data for long videos. This framework leverages
  the power of GPT-4 and text-to-image models to generate detailed scripts and
  corresponding visuals. Our approach stands out for its flexibility and
  scalability, making it a superior alternative to traditional data collection
  methods. Our extensive experiments validate that the data produced by MovieLLM
  significantly improves the performance of multimodal models in understanding
  complex video narratives, overcoming the limitations of existing datasets
  regarding scarcity and bias.
paper_ai_keywords: multimodal models synthetic high-quality data GPT-4 text-to-image models synthetic data MovieLLM flexibility scalability detailed scripts corresponding visuals adaptive computation video narratives performance of multimodal models adaptive computation regarding scarcity and bias