title: Found in the Middle: Calibrating Positional Attention Bias Improves Long
    Context Utilization
summary: Large language models (LLMs), even when specifically trained to process long
  input contexts, struggle to capture relevant information located in the middle
  of their input. This phenomenon has been known as the lost-in-the-middle
  problem. In this work, we make three contributions. First, we set out to
  understand the factors that cause this phenomenon. In doing so, we establish a
  connection between lost-in-the-middle to LLMs' intrinsic attention bias: LLMs
  exhibit a U-shaped attention bias where the tokens at the beginning and at the
  end of its input receive higher attention, regardless of their relevance.
  Second, we mitigate this positional bias through a calibration mechanism,
  found-in-the-middle, that allows the model to attend to contexts faithfully
  according to their relevance, even though when they are in the middle. Third,
  we show found-in-the-middle not only achieves better performance in locating
  relevant information within a long context, but also eventually leads to
  improved retrieval-augmented generation (RAG) performance across various tasks,
  outperforming existing methods by up to 15 percentage points. These findings
  open up future directions in understanding LLM attention bias and its potential
  consequences.
paper_authors._id: 667b2bc0c7f1674b87f472bf
paper_authors.name: Cheng-Yu Hsieh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-11T08:27:20.500Z
paper_authors._id: 6408e8663461c51cf73480e4
paper_authors.avatarUrl: /avatars/7e4cbdb4b933181c673c9dc9ad041767.svg
paper_authors.fullname: Cheng-Yu Hsieh
paper_authors.type: user
paper_authors.user: cydhsieh01
paper_authors._id: 667b2bc0c7f1674b87f472c0
paper_authors.name: Yung-Sung Chuang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-31T09:55:37.130Z
paper_authors._id: 5df84571da6d0311fd3d5407
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
paper_authors.fullname: Yung-Sung Chuang
paper_authors.type: user
paper_authors.user: voidism
paper_authors._id: 667b2bc0c7f1674b87f472c1
paper_authors.name: Chun-Liang Li
paper_authors._id: 667b2bc0c7f1674b87f472c2
paper_authors.name: Zifeng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:47:16.928Z
paper_authors._id: 64da7b3bbb090cef5559fa31
paper_authors.avatarUrl: /avatars/4eb059260f06837192c98927254cd89e.svg
paper_authors.fullname: Zifeng Wang
paper_authors.type: user
paper_authors.user: zifengw
paper_authors._id: 667b2bc0c7f1674b87f472c3
paper_authors.name: Long T. Le
paper_authors._id: 667b2bc0c7f1674b87f472c4
paper_authors.name: Abhishek Kumar
paper_authors._id: 667b2bc0c7f1674b87f472c5
paper_authors.name: James Glass
paper_authors._id: 667b2bc0c7f1674b87f472c6
paper_authors.name: Alexander Ratner
paper_authors._id: 667b2bc0c7f1674b87f472c7
paper_authors.name: Chen-Yu Lee
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-25T21:03:17.172Z
paper_authors._id: 64d1fe784dfd5df7076bfd1a
paper_authors.avatarUrl: /avatars/e2bb108d1f6b1383f0e3f263c4747b3d.svg
paper_authors.fullname: Chen-Yu Lee
paper_authors.type: user
paper_authors.user: chenyulee
paper_authors._id: 667b2bc0c7f1674b87f472c8
paper_authors.name: Ranjay Krishna
paper_authors._id: 667b2bc0c7f1674b87f472c9
paper_authors.name: Tomas Pfister
paper_title: Found in the Middle: Calibrating Positional Attention Bias Improves Long
    Context Utilization
paper_summary: Large language models (LLMs), even when specifically trained to process long
  input contexts, struggle to capture relevant information located in the middle
  of their input. This phenomenon has been known as the lost-in-the-middle
  problem. In this work, we make three contributions. First, we set out to
  understand the factors that cause this phenomenon. In doing so, we establish a
  connection between lost-in-the-middle to LLMs' intrinsic attention bias: LLMs
  exhibit a U-shaped attention bias where the tokens at the beginning and at the
  end of its input receive higher attention, regardless of their relevance.
  Second, we mitigate this positional bias through a calibration mechanism,
  found-in-the-middle, that allows the model to attend to contexts faithfully
  according to their relevance, even though when they are in the middle. Third,
  we show found-in-the-middle not only achieves better performance in locating
  relevant information within a long context, but also eventually leads to
  improved retrieval-augmented generation (RAG) performance across various tasks,
  outperforming existing methods by up to 15 percentage points. These findings
  open up future directions in understanding LLM attention bias and its potential
  consequences.
paper_ai_summary: A calibration mechanism addresses the lost-in-the-middle problem in large language models, enhancing their ability to focus on relevant information and improving retrieval-augmented generation performance.
paper_ai_keywords: large language models lost-in-the-middle problem attention bias calibration mechanism retrieval-augmented generation