title: Summarization of Multimodal Presentations with Vision-Language Models:
    Study of the Effect of Modalities and Structure
summary: Vision-Language Models (VLMs) can process visual and textual information in
  multiple formats: texts, images, interleaved texts and images, or even
  hour-long videos. In this work, we conduct fine-grained quantitative and
  qualitative analyses of automatic summarization of multimodal presentations
  using VLMs with various representations as input. From these experiments, we
  suggest cost-effective strategies for generating summaries from text-heavy
  multimodal documents under different input-length budgets using VLMs. We show
  that slides extracted from the video stream can be beneficially used as input
  against the raw video, and that a structured representation from interleaved
  slides and transcript provides the best performance. Finally, we reflect and
  comment on the nature of cross-modal interactions in multimodal presentations
  and share suggestions to improve the capabilities of VLMs to understand
  documents of this nature.
paper_authors._id: 67ff5b335cf0fe153845d1c9
paper_authors.name: Théo Gigant
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T08:42:49.462Z
paper_authors._id: 60d35154d7b174177faabd55
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1645712223620-60d35154d7b174177faabd55.jpeg
paper_authors.fullname: Théo Gigant
paper_authors.type: user
paper_authors.user: gigant
paper_authors._id: 67ff5b335cf0fe153845d1ca
paper_authors.name: Camille Guinaudeau
paper_authors._id: 67ff5b335cf0fe153845d1cb
paper_authors.name: Frédéric Dufaux
paper_title: Summarization of Multimodal Presentations with Vision-Language Models:
    Study of the Effect of Modalities and Structure
paper_summary: Vision-Language Models (VLMs) can process visual and textual information in
  multiple formats: texts, images, interleaved texts and images, or even
  hour-long videos. In this work, we conduct fine-grained quantitative and
  qualitative analyses of automatic summarization of multimodal presentations
  using VLMs with various representations as input. From these experiments, we
  suggest cost-effective strategies for generating summaries from text-heavy
  multimodal documents under different input-length budgets using VLMs. We show
  that slides extracted from the video stream can be beneficially used as input
  against the raw video, and that a structured representation from interleaved
  slides and transcript provides the best performance. Finally, we reflect and
  comment on the nature of cross-modal interactions in multimodal presentations
  and share suggestions to improve the capabilities of VLMs to understand
  documents of this nature.
paper_ai_summary: Analyses of VLMs for automatic summarization of multimodal presentations demonstrate that structured representations of interleaved slides and transcripts yield the best performance.
paper_ai_keywords: Vision-Language Models VLMs automatic summarization multimodal presentations slides transcript cross-modal interactions