title: PosterSum: A Multimodal Benchmark for Scientific Poster Summarization
summary: Generating accurate and concise textual summaries from multimodal documents
  is challenging, especially when dealing with visually complex content like
  scientific posters. We introduce PosterSum, a novel benchmark to advance the
  development of vision-language models that can understand and summarize
  scientific posters into research paper abstracts. Our dataset contains 16,305
  conference posters paired with their corresponding abstracts as summaries. Each
  poster is provided in image format and presents diverse visual understanding
  challenges, such as complex layouts, dense text regions, tables, and figures.
  We benchmark state-of-the-art Multimodal Large Language Models (MLLMs) on
  PosterSum and demonstrate that they struggle to accurately interpret and
  summarize scientific posters. We propose Segment & Summarize, a hierarchical
  method that outperforms current MLLMs on automated metrics, achieving a 3.14%
  gain in ROUGE-L. This will serve as a starting point for future research on
  poster summarization.
paper_authors._id: 67bff9608d761fc6a75e24ad
paper_authors.name: Rohit Saxena
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:13:54.284Z
paper_authors._id: 657ccbf2869d5bb0e53b482f
paper_authors.avatarUrl: /avatars/2eae5a10bdc14814a04d9f255f16de6b.svg
paper_authors.fullname: Rohit Saxena
paper_authors.type: user
paper_authors.user: rohitsaxena
paper_authors._id: 67bff9608d761fc6a75e24ae
paper_authors.name: Pasquale Minervini
paper_authors._id: 67bff9608d761fc6a75e24af
paper_authors.name: Frank Keller
paper_title: PosterSum: A Multimodal Benchmark for Scientific Poster Summarization
paper_summary: Generating accurate and concise textual summaries from multimodal documents
  is challenging, especially when dealing with visually complex content like
  scientific posters. We introduce PosterSum, a novel benchmark to advance the
  development of vision-language models that can understand and summarize
  scientific posters into research paper abstracts. Our dataset contains 16,305
  conference posters paired with their corresponding abstracts as summaries. Each
  poster is provided in image format and presents diverse visual understanding
  challenges, such as complex layouts, dense text regions, tables, and figures.
  We benchmark state-of-the-art Multimodal Large Language Models (MLLMs) on
  PosterSum and demonstrate that they struggle to accurately interpret and
  summarize scientific posters. We propose Segment & Summarize, a hierarchical
  method that outperforms current MLLMs on automated metrics, achieving a 3.14%
  gain in ROUGE-L. This will serve as a starting point for future research on
  poster summarization.
paper_ai_keywords: benchmark vision-language models scientific posters research paper abstracts dataset image format visual understanding challenges complex layouts dense text regions tables figures Multimodal Large Language Models (MLLMs) hierarchical method ROUGE-L poster summarization