title: Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports
    From Scratch with Agentic Framework
summary: Visualizations play a crucial part in effective communication of concepts and
  information. Recent advances in reasoning and retrieval augmented generation
  have enabled Large Language Models (LLMs) to perform deep research and generate
  comprehensive reports. Despite its progress, existing deep research frameworks
  primarily focus on generating text-only content, leaving the automated
  generation of interleaved texts and visualizations underexplored. This novel
  task poses key challenges in designing informative visualizations and
  effectively integrating them with text reports. To address these challenges, we
  propose Formal Description of Visualization (FDV), a structured textual
  representation of charts that enables LLMs to learn from and generate diverse,
  high-quality visualizations. Building on this representation, we introduce
  Multimodal DeepResearcher, an agentic framework that decomposes the task into
  four stages: (1) researching, (2) exemplar report textualization, (3) planning,
  and (4) multimodal report generation. For the evaluation of generated
  multimodal reports, we develop MultimodalReportBench, which contains 100
  diverse topics served as inputs along with 5 dedicated metrics. Extensive
  experiments across models and evaluation methods demonstrate the effectiveness
  of Multimodal DeepResearcher. Notably, utilizing the same Claude 3.7 Sonnet
  model, Multimodal DeepResearcher achieves an 82\% overall win rate over the
  baseline method.
paper_authors._id: 683fb5d592425f86f2be5c40
paper_authors.name: Zhaorui Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:37:44.850Z
paper_authors._id: 64a568f5764b1dce366f9fd2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a568f5764b1dce366f9fd2/9THW2AJhEVmzltEAm9mMY.jpeg
paper_authors.fullname: Zhaorui Yang
paper_authors.type: user
paper_authors.user: zhaoruiyang
paper_authors._id: 683fb5d592425f86f2be5c41
paper_authors.name: Bo Pan
paper_authors._id: 683fb5d592425f86f2be5c42
paper_authors.name: Han Wang
paper_authors._id: 683fb5d592425f86f2be5c43
paper_authors.name: Yiyao Wang
paper_authors._id: 683fb5d592425f86f2be5c44
paper_authors.name: Xingyu Liu
paper_authors._id: 683fb5d592425f86f2be5c45
paper_authors.name: Minfeng Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:37:53.514Z
paper_authors._id: 66771ddf6a685a4ffb0fbf36
paper_authors.avatarUrl: /avatars/55c9f32c710aadab0f71d53180a464a2.svg
paper_authors.fullname: Minfeng Zhu
paper_authors.type: user
paper_authors.user: twilightsnow
paper_authors._id: 683fb5d592425f86f2be5c46
paper_authors.name: Bo Zhang
paper_authors._id: 683fb5d592425f86f2be5c47
paper_authors.name: Wei Chen
paper_title: Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports
    From Scratch with Agentic Framework
paper_summary: Visualizations play a crucial part in effective communication of concepts and
  information. Recent advances in reasoning and retrieval augmented generation
  have enabled Large Language Models (LLMs) to perform deep research and generate
  comprehensive reports. Despite its progress, existing deep research frameworks
  primarily focus on generating text-only content, leaving the automated
  generation of interleaved texts and visualizations underexplored. This novel
  task poses key challenges in designing informative visualizations and
  effectively integrating them with text reports. To address these challenges, we
  propose Formal Description of Visualization (FDV), a structured textual
  representation of charts that enables LLMs to learn from and generate diverse,
  high-quality visualizations. Building on this representation, we introduce
  Multimodal DeepResearcher, an agentic framework that decomposes the task into
  four stages: (1) researching, (2) exemplar report textualization, (3) planning,
  and (4) multimodal report generation. For the evaluation of generated
  multimodal reports, we develop MultimodalReportBench, which contains 100
  diverse topics served as inputs along with 5 dedicated metrics. Extensive
  experiments across models and evaluation methods demonstrate the effectiveness
  of Multimodal DeepResearcher. Notably, utilizing the same Claude 3.7 Sonnet
  model, Multimodal DeepResearcher achieves an 82\% overall win rate over the
  baseline method.
paper_ai_summary: A new framework, Multimodal DeepResearcher, enables Large Language Models to generate high-quality multimodal reports combining text and diverse visualizations through structured textual representations.
paper_ai_keywords: Formal Description of Visualization FDV Multimodal DeepResearcher researching exemplar report textualization planning multimodal report generation MultimodalReportBench multimodal reports