The Distracting Effect: Understanding Irrelevant Passages in RAG
A well-known issue with Retrieval Augmented Generation (RAG) is that
retrieved passages that are irrelevant to the query sometimes distract the
answer-generating LLM, causing it to provide an incorrect response. In this
paper, we shed light on this core issue and formulate the distracting effect of
a passage w.r.t. a query (and an LLM). We provide a quantifiable measure of the
distracting effect of a passage and demonstrate its robustness across LLMs.
  Our research introduces novel methods for identifying and using hard
distracting passages to improve RAG systems. By fine-tuning LLMs with these
carefully selected distracting passages, we achieve up to a 7.5% increase in
answering accuracy compared to counterparts fine-tuned on conventional RAG
datasets. Our contribution is two-fold: first, we move beyond the simple binary
classification of irrelevant passages as either completely unrelated vs.
distracting, and second, we develop and analyze multiple methods for finding
hard distracting passages. To our knowledge, no other research has provided
such a comprehensive framework for identifying and utilizing hard distracting
passages.
682acf157ee0ab9140c94cb8
Chen Amiraz
682acf157ee0ab9140c94cb9
Florin Cuconasu
claimed_verified
2025-05-21T15:19:03.169Z
62bfff6788fdef8ecde8c45b
/avatars/637da88b8dce01bdc2a4c1319cc882e3.svg
Florin Cuconasu
user
florin-hf
682acf157ee0ab9140c94cba
Simone Filice
extracted_confirmed
2025-05-19T07:32:13.228Z
652028f9974423bd3ea33ae2
/avatars/50b749f9a95a3b0fb7a2bfcf17c34295.svg
Simone Filice
user
filice
682acf157ee0ab9140c94cbb
Zohar Karnin
The Distracting Effect: Understanding Irrelevant Passages in RAG
A well-known issue with Retrieval Augmented Generation (RAG) is that
retrieved passages that are irrelevant to the query sometimes distract the
answer-generating LLM, causing it to provide an incorrect response. In this
paper, we shed light on this core issue and formulate the distracting effect of
a passage w.r.t. a query (and an LLM). We provide a quantifiable measure of the
distracting effect of a passage and demonstrate its robustness across LLMs.
  Our research introduces novel methods for identifying and using hard
distracting passages to improve RAG systems. By fine-tuning LLMs with these
carefully selected distracting passages, we achieve up to a 7.5% increase in
answering accuracy compared to counterparts fine-tuned on conventional RAG
datasets. Our contribution is two-fold: first, we move beyond the simple binary
classification of irrelevant passages as either completely unrelated vs.
distracting, and second, we develop and analyze multiple methods for finding
hard distracting passages. To our knowledge, no other research has provided
such a comprehensive framework for identifying and utilizing hard distracting
passages.
Methods for identifying and utilizing hard distracting passages in Retrieval Augmented Generation (RAG) systems improve answer-generating LLMs by up to 7.5% in accuracy.
Retrieval Augmented Generation RAG answer-generating LLM distracting effect fine-tuning conventional RAG datasets binary classification hard distracting passages