title: Instruction-tuned Language Models are Better Knowledge Learners
summary: In order for large language model (LLM)-based assistants to effectively adapt
  to evolving information needs, it must be possible to update their factual
  knowledge through continued training on new data. The standard recipe for doing
  so involves continued pre-training on new documents followed by
  instruction-tuning on question-answer (QA) pairs. However, we find that LLMs
  trained with this recipe struggle to answer questions, even though the
  perplexity of documents is minimized. We found that QA pairs are generally
  straightforward, while documents are more complex, weaving many factual
  statements together in an intricate manner. Therefore, we hypothesize that it
  is beneficial to expose LLMs to QA pairs before continued pre-training on
  documents so that the process of encoding knowledge from complex documents
  takes into account how this knowledge is accessed through questions. Based on
  this, we propose pre-instruction-tuning (PIT), a method that instruction-tunes
  on questions prior to training on documents. This contrasts with standard
  instruction-tuning, which learns how to extract knowledge after training on
  documents. Extensive experiments and ablation studies demonstrate that PIT
  significantly enhances the ability of LLMs to absorb knowledge from new
  documents, outperforming standard instruction-tuning by 17.8%.
paper_authors._id: 65d58152b8c5655c03b4267f
paper_authors.name: Zhengbao Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:12:23.174Z
paper_authors._id: 6358820b27688417e37ce1ae
paper_authors.avatarUrl: /avatars/027818a4940723c406ba0d07ac940856.svg
paper_authors.fullname: Zhengbao Jiang
paper_authors.type: user
paper_authors.user: zhengbao
paper_authors._id: 65d58152b8c5655c03b42680
paper_authors.name: Zhiqing Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:12:48.636Z
paper_authors._id: 5ee24411636bdb3834e2da24
paper_authors.avatarUrl: /avatars/0e12a0b19ada4138ddf808501ba68423.svg
paper_authors.fullname: Zhiqing Sun
paper_authors.type: user
paper_authors.user: zhiqings
paper_authors._id: 65d58152b8c5655c03b42681
paper_authors.name: Weijia Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:13:20.519Z
paper_authors._id: 6400f2ed568dbe30c9161e47
paper_authors.avatarUrl: /avatars/c55938df5bce82b5d96e592a1ec36a8b.svg
paper_authors.fullname: Weijia Shi
paper_authors.type: user
paper_authors.user: swj0419
paper_authors._id: 65d58152b8c5655c03b42682
paper_authors.name: Pedro Rodriguez
paper_authors._id: 65d58152b8c5655c03b42683
paper_authors.name: Chunting Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:14:09.906Z
paper_authors._id: 64f8d07dd493d8b0d2c04ecd
paper_authors.avatarUrl: /avatars/7112232dcadcd2a7c12ec635d781b3ea.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Chunting
paper_authors._id: 65d58152b8c5655c03b42684
paper_authors.name: Graham Neubig
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:14:16.435Z
paper_authors._id: 60de14638bedd2315529d43f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1625166923504-noauth.png
paper_authors.fullname: Graham Neubig
paper_authors.type: user
paper_authors.user: gneubig
paper_authors._id: 65d58152b8c5655c03b42685
paper_authors.name: Xi Victoria Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:14:47.089Z
paper_authors._id: 61df349e34b589fbaf2e93c4
paper_authors.avatarUrl: /avatars/9dd7776d3b610d78e40c2e178ab2f112.svg
paper_authors.fullname: Victoria Lin
paper_authors.type: user
paper_authors.user: VictoriaLinML
paper_authors._id: 65d58152b8c5655c03b42686
paper_authors.name: Wen-tau Yih
paper_authors._id: 65d58152b8c5655c03b42687
paper_authors.name: Srinivasan Iyer
paper_title: Instruction-tuned Language Models are Better Knowledge Learners
paper_summary: In order for large language model (LLM)-based assistants to effectively adapt
  to evolving information needs, it must be possible to update their factual
  knowledge through continued training on new data. The standard recipe for doing
  so involves continued pre-training on new documents followed by
  instruction-tuning on question-answer (QA) pairs. However, we find that LLMs
  trained with this recipe struggle to answer questions, even though the
  perplexity of documents is minimized. We found that QA pairs are generally
  straightforward, while documents are more complex, weaving many factual
  statements together in an intricate manner. Therefore, we hypothesize that it
  is beneficial to expose LLMs to QA pairs before continued pre-training on
  documents so that the process of encoding knowledge from complex documents
  takes into account how this knowledge is accessed through questions. Based on
  this, we propose pre-instruction-tuning (PIT), a method that instruction-tunes
  on questions prior to training on documents. This contrasts with standard
  instruction-tuning, which learns how to extract knowledge after training on
  documents. Extensive experiments and ablation studies demonstrate that PIT
  significantly enhances the ability of LLMs to absorb knowledge from new
  documents, outperforming standard instruction-tuning by 17.8%.
paper_ai_summary: Pre-instruction-tuning enhances large language models' ability to learn from documents by first exposing them to question-answer pairs.
paper_ai_keywords: large language model LLM continued training pre-training instruction-tuning question-answer pairs perplexity factual knowledge pre-instruction-tuning PIT ablation studies