title: Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic
    Agents
summary: Flowcharts are a critical tool for visualizing decision-making processes.
  However, their non-linear structure and complex visual-textual relationships
  make it challenging to interpret them using LLMs, as vision-language models
  frequently hallucinate nonexistent connections and decision paths when
  analyzing these diagrams. This leads to compromised reliability for automated
  flowchart processing in critical domains such as logistics, health, and
  engineering. We introduce the task of Fine-grained Flowchart Attribution, which
  traces specific components grounding a flowchart referring LLM response.
  Flowchart Attribution ensures the verifiability of LLM predictions and improves
  explainability by linking generated responses to the flowchart's structure. We
  propose FlowPathAgent, a neurosymbolic agent that performs fine-grained post
  hoc attribution through graph-based reasoning. It first segments the flowchart,
  then converts it into a structured symbolic graph, and then employs an agentic
  approach to dynamically interact with the graph, to generate attribution paths.
  Additionally, we present FlowExplainBench, a novel benchmark for evaluating
  flowchart attributions across diverse styles, domains, and question types.
  Experimental results show that FlowPathAgent mitigates visual hallucinations in
  LLM answers over flowchart QA, outperforming strong baselines by 10-14% on our
  proposed FlowExplainBench dataset.
paper_authors._id: 6841009bdf863485e04879c8
paper_authors.name: Manan Suri
paper_authors._id: 6841009bdf863485e04879c9
paper_authors.name: Puneet Mathur
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-06-05T02:27:41.018Z
paper_authors._id: 65c16444d4c3b8dff2f0d78d
paper_authors.avatarUrl: /avatars/4ed764c1657bd260d2a12ba61c111062.svg
paper_authors.fullname: Puneet Mathur
paper_authors.type: user
paper_authors.user: puneetm
paper_authors._id: 6841009bdf863485e04879ca
paper_authors.name: Nedim Lipka
paper_authors._id: 6841009bdf863485e04879cb
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:26:52.205Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 6841009bdf863485e04879cc
paper_authors.name: Ryan A. Rossi
paper_authors._id: 6841009bdf863485e04879cd
paper_authors.name: Vivek Gupta
paper_authors._id: 6841009bdf863485e04879ce
paper_authors.name: Dinesh Manocha
paper_title: Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic
    Agents
paper_summary: Flowcharts are a critical tool for visualizing decision-making processes.
  However, their non-linear structure and complex visual-textual relationships
  make it challenging to interpret them using LLMs, as vision-language models
  frequently hallucinate nonexistent connections and decision paths when
  analyzing these diagrams. This leads to compromised reliability for automated
  flowchart processing in critical domains such as logistics, health, and
  engineering. We introduce the task of Fine-grained Flowchart Attribution, which
  traces specific components grounding a flowchart referring LLM response.
  Flowchart Attribution ensures the verifiability of LLM predictions and improves
  explainability by linking generated responses to the flowchart's structure. We
  propose FlowPathAgent, a neurosymbolic agent that performs fine-grained post
  hoc attribution through graph-based reasoning. It first segments the flowchart,
  then converts it into a structured symbolic graph, and then employs an agentic
  approach to dynamically interact with the graph, to generate attribution paths.
  Additionally, we present FlowExplainBench, a novel benchmark for evaluating
  flowchart attributions across diverse styles, domains, and question types.
  Experimental results show that FlowPathAgent mitigates visual hallucinations in
  LLM answers over flowchart QA, outperforming strong baselines by 10-14% on our
  proposed FlowExplainBench dataset.
paper_ai_summary: FlowPathAgent, a neurosymbolic agent, enhances the reliability of LLM predictions for flowchart interpretation by tracing specific components and generating accurate attribution paths.
paper_ai_keywords: Flowcharts Fine-grained Flowchart Attribution FlowPathAgent graph-based reasoning symbolic graph neurosymbolic agent flowExplainBench flowchart QA