title: GRS-QA -- Graph Reasoning-Structured Question Answering Dataset
summary: Large Language Models (LLMs) have excelled in multi-hop question-answering
  (M-QA) due to their advanced reasoning abilities. However, the impact of the
  inherent reasoning structures on LLM M-QA performance remains unclear, largely
  due to the absence of QA datasets that provide fine-grained reasoning
  structures. To address this gap, we introduce the Graph Reasoning-Structured
  Question Answering Dataset (GRS-QA), which includes both semantic contexts and
  reasoning structures for QA pairs. Unlike existing M-QA datasets, where
  different reasoning structures are entangled together, GRS-QA explicitly
  captures intricate reasoning pathways by constructing reasoning graphs, where
  nodes represent textual contexts and edges denote logical flows. These
  reasoning graphs of different structures enable a fine-grained evaluation of
  LLM reasoning capabilities across various reasoning structures. Our empirical
  analysis reveals that LLMs perform differently when handling questions with
  varying reasoning structures. This finding facilitates the exploration of
  textual structures as compared with semantics.
paper_authors._id: 67284aa68cee5fc9fcce8804
paper_authors.name: Anish Pahilajani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T21:35:11.622Z
paper_authors._id: 6392a8f56154b82600e9e6ea
paper_authors.avatarUrl: /avatars/0c2d399e42f3d4dee194d217259739fa.svg
paper_authors.fullname: Anish Pahilajani
paper_authors.type: user
paper_authors.user: Anish13
paper_authors._id: 67284aa68cee5fc9fcce8805
paper_authors.name: Devasha Trivedi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T21:35:17.358Z
paper_authors._id: 64dea11a2aeee679693748fe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64dea11a2aeee679693748fe/hlew1rksek7tmMBK5P-9g.jpeg
paper_authors.fullname: Devasha Trivedi
paper_authors.type: user
paper_authors.user: devashat
paper_authors._id: 67284aa68cee5fc9fcce8806
paper_authors.name: Jincen Shuai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T21:35:23.196Z
paper_authors._id: 65dd26e17a14b83ee0d62d8f
paper_authors.avatarUrl: /avatars/c240e81a73f4025ae3645037a0847325.svg
paper_authors.fullname: Jincen Shuai
paper_authors.type: user
paper_authors.user: afafugai
paper_authors._id: 67284aa68cee5fc9fcce8807
paper_authors.name: Khin S. Yone
paper_authors._id: 67284aa68cee5fc9fcce8808
paper_authors.name: Samyak Rajesh Jain
paper_authors._id: 67284aa68cee5fc9fcce8809
paper_authors.name: Namyong Park
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-04T04:16:42.242Z
paper_authors._id: 65f9ecb65ff1f80a69598e33
paper_authors.avatarUrl: /avatars/e0dd617533d9d7d6167a8dc4383ae08e.svg
paper_authors.fullname: Namyong Park
paper_authors.type: user
paper_authors.user: namyongp
paper_authors._id: 67284aa68cee5fc9fcce880a
paper_authors.name: Ryan A. Rossi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T21:35:48.243Z
paper_authors._id: 62a3ab83e4dd6252344d27cd
paper_authors.avatarUrl: /avatars/7ca8510f70a58dc207b104240e30c35c.svg
paper_authors.fullname: Ryan A. Rossi
paper_authors.type: user
paper_authors.user: ryanrossi
paper_authors._id: 67284aa68cee5fc9fcce880b
paper_authors.name: Nesreen K. Ahmed
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T21:35:56.985Z
paper_authors._id: 663e588ccca86c8371a913d9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663e588ccca86c8371a913d9/wgVGw5U77cqwtjovhsXyR.jpeg
paper_authors.fullname: Nesreen Ahmed
paper_authors.type: user
paper_authors.user: nkahmed
paper_authors._id: 67284aa68cee5fc9fcce880c
paper_authors.name: Franck Dernoncourt
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-11-04T04:18:15.840Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 67284aa68cee5fc9fcce880d
paper_authors.name: Yu Wang
paper_title: GRS-QA -- Graph Reasoning-Structured Question Answering Dataset
paper_summary: Large Language Models (LLMs) have excelled in multi-hop question-answering
  (M-QA) due to their advanced reasoning abilities. However, the impact of the
  inherent reasoning structures on LLM M-QA performance remains unclear, largely
  due to the absence of QA datasets that provide fine-grained reasoning
  structures. To address this gap, we introduce the Graph Reasoning-Structured
  Question Answering Dataset (GRS-QA), which includes both semantic contexts and
  reasoning structures for QA pairs. Unlike existing M-QA datasets, where
  different reasoning structures are entangled together, GRS-QA explicitly
  captures intricate reasoning pathways by constructing reasoning graphs, where
  nodes represent textual contexts and edges denote logical flows. These
  reasoning graphs of different structures enable a fine-grained evaluation of
  LLM reasoning capabilities across various reasoning structures. Our empirical
  analysis reveals that LLMs perform differently when handling questions with
  varying reasoning structures. This finding facilitates the exploration of
  textual structures as compared with semantics.
paper_ai_keywords: Large Language Models (LLMs) multi-hop question-answering (M-QA) reasoning abilities QA datasets fine-grained reasoning structures Graph Reasoning-Structured Question Answering Dataset (GRS-QA) semantic contexts reasoning structures reasoning graphs textual contexts logical flows reasoning pathways empirical analysis