title: Lost in the Middle: How Language Models Use Long Contexts
summary: While recent language models have the ability to take long contexts as input,
  relatively little is known about how well the language models use longer
  context. We analyze language model performance on two tasks that require
  identifying relevant information within their input contexts: multi-document
  question answering and key-value retrieval. We find that performance is often
  highest when relevant information occurs at the beginning or end of the input
  context, and significantly degrades when models must access relevant
  information in the middle of long contexts. Furthermore, performance
  substantially decreases as the input context grows longer, even for explicitly
  long-context models. Our analysis provides a better understanding of how
  language models use their input context and provides new evaluation protocols
  for future long-context models.
paper_authors._id: 64a784267fc46ddd14e16f2d
paper_authors.name: Nelson F. Liu
paper_authors._id: 64a784267fc46ddd14e16f2e
paper_authors.name: Kevin Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-10T09:24:26.156Z
paper_authors._id: 64a85bfad14398e270836ef8
paper_authors.avatarUrl: /avatars/61e4d298216cb89c0ec070e95d229ab6.svg
paper_authors.fullname: Kevin Lin
paper_authors.type: user
paper_authors.user: nlpkevinl
paper_authors._id: 64a784267fc46ddd14e16f2f
paper_authors.name: John Hewitt
paper_authors._id: 64a784267fc46ddd14e16f30
paper_authors.name: Ashwin Paranjape
paper_authors._id: 64a784267fc46ddd14e16f31
paper_authors.name: Michele Bevilacqua
paper_authors._id: 64a784267fc46ddd14e16f32
paper_authors.name: Fabio Petroni
paper_authors._id: 64a784267fc46ddd14e16f33
paper_authors.name: Percy Liang
paper_title: Lost in the Middle: How Language Models Use Long Contexts
paper_summary: While recent language models have the ability to take long contexts as input,
  relatively little is known about how well the language models use longer
  context. We analyze language model performance on two tasks that require
  identifying relevant information within their input contexts: multi-document
  question answering and key-value retrieval. We find that performance is often
  highest when relevant information occurs at the beginning or end of the input
  context, and significantly degrades when models must access relevant
  information in the middle of long contexts. Furthermore, performance
  substantially decreases as the input context grows longer, even for explicitly
  long-context models. Our analysis provides a better understanding of how
  language models use their input context and provides new evaluation protocols
  for future long-context models.
paper_ai_keywords: multi-document question answering key-value retrieval vari language models long-context models relevance identification input context performance beginning of the input context end of the input context multi-document question answeringing key-value retrieval long-context models relevance identification input context performance beginning of the input context end of the input context