title: Triggering Multi-Hop Reasoning for Question Answering in Language Models
    using Soft Prompts and Random Walks
summary: Despite readily memorizing world knowledge about entities, pre-trained
  language models (LMs) struggle to compose together two or more facts to perform
  multi-hop reasoning in question-answering tasks. In this work, we propose
  techniques that improve upon this limitation by relying on random walks over
  structured knowledge graphs. Specifically, we use soft prompts to guide LMs to
  chain together their encoded knowledge by learning to map multi-hop questions
  to random walk paths that lead to the answer. Applying our methods on two T5
  LMs shows substantial improvements over standard tuning approaches in answering
  questions that require 2-hop reasoning.
paper_authors._id: 64812a74cb64a2b85a44a519
paper_authors.name: Kanishka Misra
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-06-08T01:10:12.788Z
paper_authors._id: 60289f024d503386b8e75642
paper_authors.avatarUrl: /avatars/ddcb731d0abcf02933d7462b3da6f5fe.svg
paper_authors.fullname: Kanishka Misra
paper_authors.type: user
paper_authors.user: kanishka
paper_authors._id: 64812a74cb64a2b85a44a51a
paper_authors.name: Cicero Nogueira dos Santos
paper_authors._id: 64812a74cb64a2b85a44a51b
paper_authors.name: Siamak Shakeri
paper_title: Triggering Multi-Hop Reasoning for Question Answering in Language Models
    using Soft Prompts and Random Walks
paper_summary: Despite readily memorizing world knowledge about entities, pre-trained
  language models (LMs) struggle to compose together two or more facts to perform
  multi-hop reasoning in question-answering tasks. In this work, we propose
  techniques that improve upon this limitation by relying on random walks over
  structured knowledge graphs. Specifically, we use soft prompts to guide LMs to
  chain together their encoded knowledge by learning to map multi-hop questions
  to random walk paths that lead to the answer. Applying our methods on two T5
  LMs shows substantial improvements over standard tuning approaches in answering
  questions that require 2-hop reasoning.
paper_ai_keywords: pre-trained language models (LMs) multi-hop reasoning question-answering tasks random walks structured knowledge graphs soft prompts mapping multi-hop questions random walk paths 2-hop reasoning T5 LMs standard tuning approaches downstream reasoning