title: RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
summary: The reasoning steps generated by LLMs might be incomplete, as they mimic
  logical leaps common in everyday communication found in their pre-training
  data: underlying rationales are frequently left implicit (unstated). To address
  this challenge, we introduce RATIONALYST, a model for process-supervision of
  reasoning based on pre-training on a vast collection of rationale annotations
  extracted from unlabeled data. We extract 79k rationales from web-scale
  unlabelled dataset (the Pile) and a combination of reasoning datasets with
  minimal human intervention. This web-scale pre-training for reasoning allows
  RATIONALYST to consistently generalize across diverse reasoning tasks,
  including mathematical, commonsense, scientific, and logical reasoning.
  Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by
  an average of 3.9% on 7 representative reasoning benchmarks. It also
  demonstrates superior performance compared to significantly larger verifiers
  like GPT-4 and similarly sized models fine-tuned on matching training sets.
paper_authors._id: 66fe079fe9e5b5c3e49e309e
paper_authors.name: Dongwei Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:15:24.379Z
paper_authors._id: 63eba2774dcaaf087638e3d6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676386919413-noauth.jpeg
paper_authors.fullname: Jiang
paper_authors.type: user
paper_authors.user: Dongwei
paper_authors._id: 66fe079fe9e5b5c3e49e309f
paper_authors.name: Guoxuan Wang
paper_authors._id: 66fe079fe9e5b5c3e49e30a0
paper_authors.name: Yining Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:22:30.977Z
paper_authors._id: 642f742270daaa6e7209a2c8
paper_authors.avatarUrl: /avatars/746fb840c220329f69a17905ea519322.svg
paper_authors.fullname: Yining Lu
paper_authors.type: user
paper_authors.user: ylu610
paper_authors._id: 66fe079fe9e5b5c3e49e30a1
paper_authors.name: Andrew Wang
paper_authors._id: 66fe079fe9e5b5c3e49e30a2
paper_authors.name: Jingyu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T07:32:14.554Z
paper_authors._id: 62fb40b59af1d16bc0ac60f4
paper_authors.avatarUrl: /avatars/03ff66a419db8f2bc8e89a3b47aaaeac.svg
paper_authors.fullname: Jack Zhang
paper_authors.type: user
paper_authors.user: jackzhang
paper_authors._id: 66fe079fe9e5b5c3e49e30a3
paper_authors.name: Chuyu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:20:21.699Z
paper_authors._id: 66240a576046dd4b8586a604
paper_authors.avatarUrl: /avatars/6b4c5fbdaafd6f42d8f4581d0f48d2c0.svg
paper_authors.fullname: Chuyu Liu
paper_authors.type: user
paper_authors.user: ChuyuLiu
paper_authors._id: 66fe079fe9e5b5c3e49e30a4
paper_authors.name: Benjamin Van Durme
paper_authors._id: 66fe079fe9e5b5c3e49e30a5
paper_authors.name: Daniel Khashabi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T09:20:05.299Z
paper_authors._id: 5f6540c65e78cc6b0ed3199d
paper_authors.avatarUrl: /avatars/0280d4df417855965a0964d22766c012.svg
paper_authors.fullname: Daniel Khashabi
paper_authors.type: user
paper_authors.user: danyaljj
paper_title: RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
paper_summary: The reasoning steps generated by LLMs might be incomplete, as they mimic
  logical leaps common in everyday communication found in their pre-training
  data: underlying rationales are frequently left implicit (unstated). To address
  this challenge, we introduce RATIONALYST, a model for process-supervision of
  reasoning based on pre-training on a vast collection of rationale annotations
  extracted from unlabeled data. We extract 79k rationales from web-scale
  unlabelled dataset (the Pile) and a combination of reasoning datasets with
  minimal human intervention. This web-scale pre-training for reasoning allows
  RATIONALYST to consistently generalize across diverse reasoning tasks,
  including mathematical, commonsense, scientific, and logical reasoning.
  Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by
  an average of 3.9% on 7 representative reasoning benchmarks. It also
  demonstrates superior performance compared to significantly larger verifiers
  like GPT-4 and similarly sized models fine-tuned on matching training sets.
paper_ai_keywords: process-supervision reason LLMs pre-training rationale Pile reasoning datasets reasoning benchmarks LLaMa-3-8B