ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual
  Attribution
Large Language Models (LLMs) can perform chart question-answering tasks but
often generate unverified hallucinated responses. Existing answer attribution
methods struggle to ground responses in source charts due to limited
visual-semantic context, complex visual-text alignment requirements, and
difficulties in bounding box prediction across complex layouts. We present
ChartCitor, a multi-agent framework that provides fine-grained bounding box
citations by identifying supporting evidence within chart images. The system
orchestrates LLM agents to perform chart-to-table extraction, answer
reformulation, table augmentation, evidence retrieval through pre-filtering and
re-ranking, and table-to-chart mapping. ChartCitor outperforms existing
baselines across different chart types. Qualitative user studies show that
ChartCitor helps increase user trust in Generative AI by providing enhanced
explainability for LLM-assisted chart QA and enables professionals to be more
productive.
67a5c7601e6db426653ebc3d
Kanika Goswami
67a5c7601e6db426653ebc3e
Puneet Mathur
admin_assigned
2025-02-07T10:47:20.093Z
65c16444d4c3b8dff2f0d78d
/avatars/4ed764c1657bd260d2a12ba61c111062.svg
Puneet Mathur
user
puneetm
67a5c7601e6db426653ebc3f
Ryan Rossi
admin_assigned
2025-02-07T10:47:32.496Z
62a3ab83e4dd6252344d27cd
/avatars/7ca8510f70a58dc207b104240e30c35c.svg
Ryan A. Rossi
user
ryanrossi
67a5c7601e6db426653ebc40
Franck Dernoncourt
claimed_verified
2025-02-07T10:30:50.575Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual
  Attribution
Large Language Models (LLMs) can perform chart question-answering tasks but
often generate unverified hallucinated responses. Existing answer attribution
methods struggle to ground responses in source charts due to limited
visual-semantic context, complex visual-text alignment requirements, and
difficulties in bounding box prediction across complex layouts. We present
ChartCitor, a multi-agent framework that provides fine-grained bounding box
citations by identifying supporting evidence within chart images. The system
orchestrates LLM agents to perform chart-to-table extraction, answer
reformulation, table augmentation, evidence retrieval through pre-filtering and
re-ranking, and table-to-chart mapping. ChartCitor outperforms existing
baselines across different chart types. Qualitative user studies show that
ChartCitor helps increase user trust in Generative AI by providing enhanced
explainability for LLM-assisted chart QA and enables professionals to be more
productive.
ChartCitor, a multi-agent framework, enhances chart question-answering by providing fine-grained bounding box citations, improving user trust and productivity in Generative AI applications.
ChartCitor multi-agent framework bounding box citations chart-to-table extraction answer reformulation table augmentation evidence retrieval pre-filtering re-ranking table-to-chart mapping chart question-answering user trust explainability