title: ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual
    Attribution
summary: Large Language Models (LLMs) can perform chart question-answering tasks but
  often generate unverified hallucinated responses. Existing answer attribution
  methods struggle to ground responses in source charts due to limited
  visual-semantic context, complex visual-text alignment requirements, and
  difficulties in bounding box prediction across complex layouts. We present
  ChartCitor, a multi-agent framework that provides fine-grained bounding box
  citations by identifying supporting evidence within chart images. The system
  orchestrates LLM agents to perform chart-to-table extraction, answer
  reformulation, table augmentation, evidence retrieval through pre-filtering and
  re-ranking, and table-to-chart mapping. ChartCitor outperforms existing
  baselines across different chart types. Qualitative user studies show that
  ChartCitor helps increase user trust in Generative AI by providing enhanced
  explainability for LLM-assisted chart QA and enables professionals to be more
  productive.
paper_authors._id: 67a5c7601e6db426653ebc3d
paper_authors.name: Kanika Goswami
paper_authors._id: 67a5c7601e6db426653ebc3e
paper_authors.name: Puneet Mathur
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:47:20.093Z
paper_authors._id: 65c16444d4c3b8dff2f0d78d
paper_authors.avatarUrl: /avatars/4ed764c1657bd260d2a12ba61c111062.svg
paper_authors.fullname: Puneet Mathur
paper_authors.type: user
paper_authors.user: puneetm
paper_authors._id: 67a5c7601e6db426653ebc3f
paper_authors.name: Ryan Rossi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-07T10:47:32.496Z
paper_authors._id: 62a3ab83e4dd6252344d27cd
paper_authors.avatarUrl: /avatars/7ca8510f70a58dc207b104240e30c35c.svg
paper_authors.fullname: Ryan A. Rossi
paper_authors.type: user
paper_authors.user: ryanrossi
paper_authors._id: 67a5c7601e6db426653ebc40
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T10:30:50.575Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_title: ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual
    Attribution
paper_summary: Large Language Models (LLMs) can perform chart question-answering tasks but
  often generate unverified hallucinated responses. Existing answer attribution
  methods struggle to ground responses in source charts due to limited
  visual-semantic context, complex visual-text alignment requirements, and
  difficulties in bounding box prediction across complex layouts. We present
  ChartCitor, a multi-agent framework that provides fine-grained bounding box
  citations by identifying supporting evidence within chart images. The system
  orchestrates LLM agents to perform chart-to-table extraction, answer
  reformulation, table augmentation, evidence retrieval through pre-filtering and
  re-ranking, and table-to-chart mapping. ChartCitor outperforms existing
  baselines across different chart types. Qualitative user studies show that
  ChartCitor helps increase user trust in Generative AI by providing enhanced
  explainability for LLM-assisted chart QA and enables professionals to be more
  productive.
paper_ai_summary: ChartCitor, a multi-agent framework, enhances chart question-answering by providing fine-grained bounding box citations, improving user trust and productivity in Generative AI applications.
paper_ai_keywords: ChartCitor multi-agent framework bounding box citations chart-to-table extraction answer reformulation table augmentation evidence retrieval pre-filtering re-ranking table-to-chart mapping chart question-answering user trust explainability