title: Scaling Reasoning can Improve Factuality in Large Language Models
summary: Recent studies on large language model (LLM) reasoning capabilities have
  demonstrated promising improvements in model performance by leveraging a
  lengthy thinking process and additional computational resources during
  inference, primarily in tasks involving mathematical reasoning (Muennighoff et
  al., 2025). However, it remains uncertain if longer reasoning chains inherently
  enhance factual accuracy, particularly beyond mathematical contexts. In this
  work, we thoroughly examine LLM reasoning within complex open-domain
  question-answering (QA) scenarios. We initially distill reasoning traces from
  advanced, large-scale reasoning models (QwQ-32B and DeepSeek-R1-671B), then
  fine-tune a variety of models ranging from smaller, instruction-tuned variants
  to larger architectures based on Qwen2.5. To enrich reasoning traces, we
  introduce factual information from knowledge graphs in the form of paths into
  our reasoning traces. Our experimental setup includes four baseline approaches
  and six different instruction-tuned models evaluated across a benchmark of six
  datasets, encompassing over 22.6K questions. Overall, we carry out 168
  experimental runs and analyze approximately 1.7 million reasoning traces. Our
  findings indicate that, within a single run, smaller reasoning models achieve
  noticeable improvements in factual accuracy compared to their original
  instruction-tuned counterparts. Moreover, our analysis demonstrates that adding
  test-time compute and token budgets factual accuracy consistently improves by
  2-8%, further confirming the effectiveness of test-time scaling for enhancing
  performance and consequently improving reasoning accuracy in open-domain QA
  tasks. We release all the experimental artifacts for further research.
paper_authors._id: 682ad417500638b80a43471d
paper_authors.name: Mike Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-19T07:23:04.053Z
paper_authors._id: 60d33fbbd7b174177faabd4f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60d33fbbd7b174177faabd4f/pfyv_xj2B2m2N4F4sT9zJ.jpeg
paper_authors.fullname: Mike Zhang
paper_authors.type: user
paper_authors.user: jjzha
paper_authors._id: 682ad417500638b80a43471e
paper_authors.name: Johannes Bjerva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-19T07:25:58.827Z
paper_authors._id: 678fa79005ae7fe48d03ba47
paper_authors.avatarUrl: /avatars/a78ab2b37fa3e18ace783f6f71f5a361.svg
paper_authors.fullname: Johannes Bjerva
paper_authors.type: user
paper_authors.user: bjerva
paper_authors._id: 682ad417500638b80a43471f
paper_authors.name: Russa Biswas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-19T07:26:04.749Z
paper_authors._id: 60ed4c56abab3c2620df8ac8
paper_authors.avatarUrl: /avatars/ad5508c1c94a96f6d1290e4735e81b73.svg
paper_authors.fullname: Russa Biswas
paper_authors.type: user
paper_authors.user: rubis
paper_title: Scaling Reasoning can Improve Factuality in Large Language Models
paper_summary: Recent studies on large language model (LLM) reasoning capabilities have
  demonstrated promising improvements in model performance by leveraging a
  lengthy thinking process and additional computational resources during
  inference, primarily in tasks involving mathematical reasoning (Muennighoff et
  al., 2025). However, it remains uncertain if longer reasoning chains inherently
  enhance factual accuracy, particularly beyond mathematical contexts. In this
  work, we thoroughly examine LLM reasoning within complex open-domain
  question-answering (QA) scenarios. We initially distill reasoning traces from
  advanced, large-scale reasoning models (QwQ-32B and DeepSeek-R1-671B), then
  fine-tune a variety of models ranging from smaller, instruction-tuned variants
  to larger architectures based on Qwen2.5. To enrich reasoning traces, we
  introduce factual information from knowledge graphs in the form of paths into
  our reasoning traces. Our experimental setup includes four baseline approaches
  and six different instruction-tuned models evaluated across a benchmark of six
  datasets, encompassing over 22.6K questions. Overall, we carry out 168
  experimental runs and analyze approximately 1.7 million reasoning traces. Our
  findings indicate that, within a single run, smaller reasoning models achieve
  noticeable improvements in factual accuracy compared to their original
  instruction-tuned counterparts. Moreover, our analysis demonstrates that adding
  test-time compute and token budgets factual accuracy consistently improves by
  2-8%, further confirming the effectiveness of test-time scaling for enhancing
  performance and consequently improving reasoning accuracy in open-domain QA
  tasks. We release all the experimental artifacts for further research.
paper_ai_summary: Enhancing factual accuracy in open-domain QA through reasoning trace distillation, knowledge graph integration, and test-time scaling improves smaller models' performance across various datasets.
paper_ai_keywords: large language model (LLM) reasoning capabilities length thinking process computational resources mathematical reasoning complex open-domain question-answering (QA) scenarios reasoning traces QwQ-32B DeepSeek-R1-671B instruction-tuned Qwen2.5 knowledge graphs factual accuracy test-time compute token budgets reasoning accuracy