title: RAFT: Adapting Language Model to Domain Specific RAG
summary: Pretraining Large Language Models (LLMs) on large corpora of textual data is
  now a standard paradigm. When using these LLMs for many downstream
  applications, it is common to additionally bake in new knowledge (e.g.,
  time-critical news, or private domain knowledge) into the pretrained model
  either through RAG-based-prompting, or fine-tuning. However, the optimal
  methodology for the model to gain such new knowledge remains an open question.
  In this paper, we present Retrieval Augmented FineTuning (RAFT), a training
  recipe that improves the model's ability to answer questions in a "open-book"
  in-domain settings. In RAFT, given a question, and a set of retrieved
  documents, we train the model to ignore those documents that don't help in
  answering the question, which we call, distractor documents. RAFT accomplishes
  this by citing verbatim the right sequence from the relevant document that
  would help answer the question. This coupled with RAFT's chain-of-thought-style
  response helps improve the model's ability to reason. In domain-specific RAG,
  RAFT consistently improves the model's performance across PubMed, HotpotQA, and
  Gorilla datasets, presenting a post-training recipe to improve pre-trained LLMs
  to in-domain RAG. RAFT's code and demo are open-sourced at
  github.com/ShishirPatil/gorilla.
paper_authors._id: 65f7afd899c842dd937c831a
paper_authors.name: Tianjun Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-01T05:49:19.111Z
paper_authors._id: 6374cd6b6ea8da14f8fef8dc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6374cd6b6ea8da14f8fef8dc/l13bg0tKDjCnUw3I895QZ.png
paper_authors.fullname: Tianjun Zhang
paper_authors.type: user
paper_authors.user: tianjunz
paper_authors._id: 65f7afd899c842dd937c831b
paper_authors.name: Shishir G. Patil
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-20T16:30:17.252Z
paper_authors._id: 6471b9f6094820190c324eec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6471b9f6094820190c324eec/yb62BiXzRFY1_pS0GAMGq.jpeg
paper_authors.fullname: Shishir Patil
paper_authors.type: user
paper_authors.user: shishirpatil
paper_authors._id: 65f7afd899c842dd937c831c
paper_authors.name: Naman Jain
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-19T08:45:51.186Z
paper_authors._id: 6411ed451e42164b9f0f8b24
paper_authors.avatarUrl: /avatars/a5c023f407894d3d1dba7b343b847380.svg
paper_authors.fullname: Naman Jain
paper_authors.type: user
paper_authors.user: StringChaos
paper_authors._id: 65f7afd899c842dd937c831d
paper_authors.name: Sheng Shen
paper_authors._id: 65f7afd899c842dd937c831e
paper_authors.name: Matei Zaharia
paper_authors._id: 65f7afd899c842dd937c831f
paper_authors.name: Ion Stoica
paper_authors._id: 65f7afd899c842dd937c8320
paper_authors.name: Joseph E. Gonzalez
paper_title: RAFT: Adapting Language Model to Domain Specific RAG
paper_summary: Pretraining Large Language Models (LLMs) on large corpora of textual data is
  now a standard paradigm. When using these LLMs for many downstream
  applications, it is common to additionally bake in new knowledge (e.g.,
  time-critical news, or private domain knowledge) into the pretrained model
  either through RAG-based-prompting, or fine-tuning. However, the optimal
  methodology for the model to gain such new knowledge remains an open question.
  In this paper, we present Retrieval Augmented FineTuning (RAFT), a training
  recipe that improves the model's ability to answer questions in a "open-book"
  in-domain settings. In RAFT, given a question, and a set of retrieved
  documents, we train the model to ignore those documents that don't help in
  answering the question, which we call, distractor documents. RAFT accomplishes
  this by citing verbatim the right sequence from the relevant document that
  would help answer the question. This coupled with RAFT's chain-of-thought-style
  response helps improve the model's ability to reason. In domain-specific RAG,
  RAFT consistently improves the model's performance across PubMed, HotpotQA, and
  Gorilla datasets, presenting a post-training recipe to improve pre-trained LLMs
  to in-domain RAG. RAFT's code and demo are open-sourced at
  github.com/ShishirPatil/gorilla.
paper_ai_summary: Retrieval Augmented FineTuning (RAFT) enhances pre-trained large language models' in-domain performance by filtering out irrelevant documents and citing relevant passages.
paper_ai_keywords: Retrieval Augmented FineTuning RAFT distractor documents chain-of-thought-style PubMed HotpotQA Gorilla datasets