title: PlotGen: Multi-Agent LLM-based Scientific Data Visualization via
    Multimodal Feedback
summary: Scientific data visualization is pivotal for transforming raw data into
  comprehensible visual representations, enabling pattern recognition,
  forecasting, and the presentation of data-driven insights. However, novice
  users often face difficulties due to the complexity of selecting appropriate
  tools and mastering visualization techniques. Large Language Models (LLMs) have
  recently demonstrated potential in assisting code generation, though they
  struggle with accuracy and require iterative debugging. In this paper, we
  propose PlotGen, a novel multi-agent framework aimed at automating the creation
  of precise scientific visualizations. PlotGen orchestrates multiple LLM-based
  agents, including a Query Planning Agent that breaks down complex user requests
  into executable steps, a Code Generation Agent that converts pseudocode into
  executable Python code, and three retrieval feedback agents - a Numeric
  Feedback Agent, a Lexical Feedback Agent, and a Visual Feedback Agent - that
  leverage multimodal LLMs to iteratively refine the data accuracy, textual
  labels, and visual correctness of generated plots via self-reflection.
  Extensive experiments show that PlotGen outperforms strong baselines, achieving
  a 4-6 percent improvement on the MatPlotBench dataset, leading to enhanced user
  trust in LLM-generated visualizations and improved novice productivity due to a
  reduction in debugging time needed for plot errors.
paper_authors._id: 67a5e076b94446dfc848533b
paper_authors.name: Kanika Goswami
paper_authors._id: 67a5e076b94446dfc848533c
paper_authors.name: Puneet Mathur
paper_authors._id: 67a5e076b94446dfc848533d
paper_authors.name: Ryan Rossi
paper_authors._id: 67a5e076b94446dfc848533e
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T10:30:03.421Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_title: PlotGen: Multi-Agent LLM-based Scientific Data Visualization via
    Multimodal Feedback
paper_summary: Scientific data visualization is pivotal for transforming raw data into
  comprehensible visual representations, enabling pattern recognition,
  forecasting, and the presentation of data-driven insights. However, novice
  users often face difficulties due to the complexity of selecting appropriate
  tools and mastering visualization techniques. Large Language Models (LLMs) have
  recently demonstrated potential in assisting code generation, though they
  struggle with accuracy and require iterative debugging. In this paper, we
  propose PlotGen, a novel multi-agent framework aimed at automating the creation
  of precise scientific visualizations. PlotGen orchestrates multiple LLM-based
  agents, including a Query Planning Agent that breaks down complex user requests
  into executable steps, a Code Generation Agent that converts pseudocode into
  executable Python code, and three retrieval feedback agents - a Numeric
  Feedback Agent, a Lexical Feedback Agent, and a Visual Feedback Agent - that
  leverage multimodal LLMs to iteratively refine the data accuracy, textual
  labels, and visual correctness of generated plots via self-reflection.
  Extensive experiments show that PlotGen outperforms strong baselines, achieving
  a 4-6 percent improvement on the MatPlotBench dataset, leading to enhanced user
  trust in LLM-generated visualizations and improved novice productivity due to a
  reduction in debugging time needed for plot errors.
paper_ai_summary: PlotGen is a multi-agent framework that automates scientific visualization by coordinating LLM-based agents to generate accurate, reliable plots, reducing user debugging time and enhancing novice productivity.
paper_ai_keywords: PlotGen multi-agent framework large language models LLM-based agents Query Planning Agent Code Generation Agent Numeric Feedback Agent Lexical Feedback Agent Visual Feedback Agent multimodal LLMs self-reflection MatPlotBench