title: ARR: Question Answering with Large Language Models via Analyzing,
    Retrieving, and Reasoning
summary: Large language models (LLMs) achieve remarkable performance on challenging
  benchmarks that are often structured as multiple-choice question-answering (QA)
  tasks. Zero-shot Chain-of-Thought (CoT) prompting enhances reasoning in LLMs
  but provides only vague and generic guidance ("think step by step"). This paper
  introduces ARR, an intuitive and effective zero-shot prompting method that
  explicitly incorporates three key steps in QA solving: analyzing the intent of
  the question, retrieving relevant information, and reasoning step by step.
  Comprehensive experiments across diverse and challenging QA tasks demonstrate
  that ARR consistently improves the Baseline (without ARR prompting) and
  outperforms CoT. Ablation and case studies further validate the positive
  contributions of each component: analyzing, retrieving, and reasoning. Notably,
  intent analysis plays a vital role in ARR. Additionally, extensive evaluations
  across various model sizes, LLM series, and generation settings solidify the
  effectiveness, robustness, and generalizability of ARR.
paper_authors._id: 67a9b911b1f5eece682d7961
paper_authors.name: Yuwei Yin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T09:49:32.672Z
paper_authors._id: 64510a21f800611f94f0d9f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/lOeHK9Bvt3IXcB7Urx6jZ.jpeg
paper_authors.fullname: Yuwei Yin
paper_authors.type: user
paper_authors.user: yuweiyin
paper_authors._id: 67a9b911b1f5eece682d7962
paper_authors.name: Giuseppe Carenini
paper_title: ARR: Question Answering with Large Language Models via Analyzing,
    Retrieving, and Reasoning
paper_summary: Large language models (LLMs) achieve remarkable performance on challenging
  benchmarks that are often structured as multiple-choice question-answering (QA)
  tasks. Zero-shot Chain-of-Thought (CoT) prompting enhances reasoning in LLMs
  but provides only vague and generic guidance ("think step by step"). This paper
  introduces ARR, an intuitive and effective zero-shot prompting method that
  explicitly incorporates three key steps in QA solving: analyzing the intent of
  the question, retrieving relevant information, and reasoning step by step.
  Comprehensive experiments across diverse and challenging QA tasks demonstrate
  that ARR consistently improves the Baseline (without ARR prompting) and
  outperforms CoT. Ablation and case studies further validate the positive
  contributions of each component: analyzing, retrieving, and reasoning. Notably,
  intent analysis plays a vital role in ARR. Additionally, extensive evaluations
  across various model sizes, LLM series, and generation settings solidify the
  effectiveness, robustness, and generalizability of ARR.
paper_ai_summary: ARR, a zero-shot prompting method for question-answering tasks, incorporates intent analysis, retrieval, and reasoning to enhance performance beyond zero-shot Chain-of-Thought prompting.
paper_ai_keywords: Large language models zero-shot Chain-of-Thought zero-shot prompting intent analysis retrieval reasoning question-answering comprehensive experiments ablation studies case studies model sizes LLM series generation settings