title: Visually Interpretable Subtask Reasoning for Visual Question Answering
summary: Answering complex visual questions like `Which red furniture can be used for
  sitting?' requires multi-step reasoning, including object recognition,
  attribute filtering, and relational understanding. Recent work improves
  interpretability in multimodal large language models (MLLMs) by decomposing
  tasks into sub-task programs, but these methods are computationally expensive
  and less accurate due to poor adaptation to target data. To address this, we
  introduce VISTAR (Visually Interpretable Subtask-Aware Reasoning Model), a
  subtask-driven training framework that enhances both interpretability and
  reasoning by generating textual and visual explanations within MLLMs. Instead
  of relying on external models, VISTAR fine-tunes MLLMs to produce structured
  Subtask-of-Thought rationales (step-by-step reasoning sequences). Experiments
  on two benchmarks show that VISTAR consistently improves reasoning accuracy
  while maintaining interpretability. Our code and dataset will be available at
  https://github.com/ChengJade/VISTAR.
paper_authors._id: 6824bf46b4f1e881646d2dda
paper_authors.name: Yu Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-15T10:32:11.536Z
paper_authors._id: 65a84987c5ffe1d019d7ac02
paper_authors.avatarUrl: /avatars/1e54a4d99e98da142f0d6441df3d2d4b.svg
paper_authors.fullname: Yu Cheng
paper_authors.type: user
paper_authors.user: JadeCheng
paper_authors._id: 6824bf46b4f1e881646d2ddb
paper_authors.name: Arushi Goel
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-14T16:05:27.452Z
paper_authors._id: 65aa4c2ea92a64ef5b2f2ff9
paper_authors.avatarUrl: /avatars/13d894fe51c011620d094c2f1a114aaf.svg
paper_authors.fullname: Arushi Goel
paper_authors.type: user
paper_authors.user: goarushi
paper_authors._id: 6824bf46b4f1e881646d2ddc
paper_authors.name: Hakan Bilen
paper_title: Visually Interpretable Subtask Reasoning for Visual Question Answering
paper_summary: Answering complex visual questions like `Which red furniture can be used for
  sitting?' requires multi-step reasoning, including object recognition,
  attribute filtering, and relational understanding. Recent work improves
  interpretability in multimodal large language models (MLLMs) by decomposing
  tasks into sub-task programs, but these methods are computationally expensive
  and less accurate due to poor adaptation to target data. To address this, we
  introduce VISTAR (Visually Interpretable Subtask-Aware Reasoning Model), a
  subtask-driven training framework that enhances both interpretability and
  reasoning by generating textual and visual explanations within MLLMs. Instead
  of relying on external models, VISTAR fine-tunes MLLMs to produce structured
  Subtask-of-Thought rationales (step-by-step reasoning sequences). Experiments
  on two benchmarks show that VISTAR consistently improves reasoning accuracy
  while maintaining interpretability. Our code and dataset will be available at
  https://github.com/ChengJade/VISTAR.
paper_ai_summary: VISTAR enhances multimodal large language models by generating structured reasoning sequences to improve accuracy and interpretability.
paper_ai_keywords: multimodal large language models subtask-driven training textual explanations visual explanations Subtask-of-Thought rationales step-by-step reasoning sequences