Triggering Multi-Hop Reasoning for Question Answering in Language Models
  using Soft Prompts and Random Walks
Despite readily memorizing world knowledge about entities, pre-trained
language models (LMs) struggle to compose together two or more facts to perform
multi-hop reasoning in question-answering tasks. In this work, we propose
techniques that improve upon this limitation by relying on random walks over
structured knowledge graphs. Specifically, we use soft prompts to guide LMs to
chain together their encoded knowledge by learning to map multi-hop questions
to random walk paths that lead to the answer. Applying our methods on two T5
LMs shows substantial improvements over standard tuning approaches in answering
questions that require 2-hop reasoning.
64812a74cb64a2b85a44a519
Kanishka Misra
extracted_pending
2023-06-08T01:10:12.788Z
60289f024d503386b8e75642
/avatars/ddcb731d0abcf02933d7462b3da6f5fe.svg
Kanishka Misra
user
kanishka
64812a74cb64a2b85a44a51a
Cicero Nogueira dos Santos
64812a74cb64a2b85a44a51b
Siamak Shakeri
Triggering Multi-Hop Reasoning for Question Answering in Language Models
  using Soft Prompts and Random Walks
Despite readily memorizing world knowledge about entities, pre-trained
language models (LMs) struggle to compose together two or more facts to perform
multi-hop reasoning in question-answering tasks. In this work, we propose
techniques that improve upon this limitation by relying on random walks over
structured knowledge graphs. Specifically, we use soft prompts to guide LMs to
chain together their encoded knowledge by learning to map multi-hop questions
to random walk paths that lead to the answer. Applying our methods on two T5
LMs shows substantial improvements over standard tuning approaches in answering
questions that require 2-hop reasoning.
Soft prompts enhance multi-hop reasoning in pre-trained language models by guiding them to follow random walk paths on structured knowledge graphs.
pre-trained language models multi-hop reasoning question-answering tasks soft prompts random walks structured knowledge graphs T5 LMs