title: ChartLens: Fine-grained Visual Attribution in Charts
summary: The growing capabilities of multimodal large language models (MLLMs) have
  advanced tasks like chart understanding. However, these models often suffer
  from hallucinations, where generated text sequences conflict with the provided
  visual data. To address this, we introduce Post-Hoc Visual Attribution for
  Charts, which identifies fine-grained chart elements that validate a given
  chart-associated response. We propose ChartLens, a novel chart attribution
  algorithm that uses segmentation-based techniques to identify chart objects and
  employs set-of-marks prompting with MLLMs for fine-grained visual attribution.
  Additionally, we present ChartVA-Eval, a benchmark with synthetic and
  real-world charts from diverse domains like finance, policy, and economics,
  featuring fine-grained attribution annotations. Our evaluations show that
  ChartLens improves fine-grained attributions by 26-66%.
paper_authors._id: 68392279896eb9ceb71fac39
paper_authors.name: Manan Suri
paper_authors._id: 68392279896eb9ceb71fac3a
paper_authors.name: Puneet Mathur
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-30T03:14:02.414Z
paper_authors._id: 65c16444d4c3b8dff2f0d78d
paper_authors.avatarUrl: /avatars/4ed764c1657bd260d2a12ba61c111062.svg
paper_authors.fullname: Puneet Mathur
paper_authors.type: user
paper_authors.user: puneetm
paper_authors._id: 68392279896eb9ceb71fac3b
paper_authors.name: Nedim Lipka
paper_authors._id: 68392279896eb9ceb71fac3c
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:28.558Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 68392279896eb9ceb71fac3d
paper_authors.name: Ryan A. Rossi
paper_authors._id: 68392279896eb9ceb71fac3e
paper_authors.name: Dinesh Manocha
paper_title: ChartLens: Fine-grained Visual Attribution in Charts
paper_summary: The growing capabilities of multimodal large language models (MLLMs) have
  advanced tasks like chart understanding. However, these models often suffer
  from hallucinations, where generated text sequences conflict with the provided
  visual data. To address this, we introduce Post-Hoc Visual Attribution for
  Charts, which identifies fine-grained chart elements that validate a given
  chart-associated response. We propose ChartLens, a novel chart attribution
  algorithm that uses segmentation-based techniques to identify chart objects and
  employs set-of-marks prompting with MLLMs for fine-grained visual attribution.
  Additionally, we present ChartVA-Eval, a benchmark with synthetic and
  real-world charts from diverse domains like finance, policy, and economics,
  featuring fine-grained attribution annotations. Our evaluations show that
  ChartLens improves fine-grained attributions by 26-66%.
paper_ai_summary: ChartLens enhances multimodal language models with fine-grained visual attributions, improving the accuracy of chart understanding by 26-66%.
paper_ai_keywords: multimodal large language models MLLMs chart understanding hallucinations Post-Hoc Visual Attribution ChartLens segmentation-based techniques set-of-marks prompting ChartVA-Eval synthetic charts real-world charts fine-grained attribution annotations