SAGE: A Framework of Precise Retrieval for RAG
Retrieval-augmented generation (RAG) has demonstrated significant proficiency
in conducting question-answering (QA) tasks within a specified corpus.
Nonetheless, numerous failure instances of RAG in QA still exist. These
failures are not solely attributable to the limitations of Large Language
Models (LLMs); instead, they predominantly arise from the retrieval of
inaccurate information for LLMs due to two limitations: (1) Current RAG methods
segment the corpus without considering semantics, making it difficult to find
relevant context due to impaired correlation between questions and the
segments. (2) There is a trade-off between missing essential context with fewer
context retrieved and getting irrelevant context with more context retrieved.
  In this paper, we introduce a RAG framework (SAGE), to overcome these
limitations. First, to address the segmentation issue without considering
semantics, we propose to train a semantic segmentation model. This model is
trained to segment the corpus into semantically complete chunks. Second, to
ensure that only the most relevant chunks are retrieved while the irrelevant
ones are ignored, we design a chunk selection algorithm to dynamically select
chunks based on the decreasing speed of the relevance score, leading to a more
relevant selection. Third, to further ensure the precision of the retrieved
chunks, we propose letting LLMs assess whether retrieved chunks are excessive
or lacking and then adjust the amount of context accordingly. Experiments show
that SAGE outperforms baselines by 61.25% in the quality of QA on average.
Moreover, by avoiding retrieving noisy context, SAGE lowers the cost of the
tokens consumed in LLM inference and achieves a 49.41% enhancement in cost
efficiency on average. Additionally, our work offers valuable insights for
boosting RAG.
67c75e18cb29e2a4b0eb0293
Jintao Zhang
claimed_verified
2025-03-07T09:26:50.367Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
user
jt-zhang
67c75e18cb29e2a4b0eb0294
Guoliang Li
67c75e18cb29e2a4b0eb0295
Jinyang Su
SAGE: A Framework of Precise Retrieval for RAG
Retrieval-augmented generation (RAG) has demonstrated significant proficiency
in conducting question-answering (QA) tasks within a specified corpus.
Nonetheless, numerous failure instances of RAG in QA still exist. These
failures are not solely attributable to the limitations of Large Language
Models (LLMs); instead, they predominantly arise from the retrieval of
inaccurate information for LLMs due to two limitations: (1) Current RAG methods
segment the corpus without considering semantics, making it difficult to find
relevant context due to impaired correlation between questions and the
segments. (2) There is a trade-off between missing essential context with fewer
context retrieved and getting irrelevant context with more context retrieved.
  In this paper, we introduce a RAG framework (SAGE), to overcome these
limitations. First, to address the segmentation issue without considering
semantics, we propose to train a semantic segmentation model. This model is
trained to segment the corpus into semantically complete chunks. Second, to
ensure that only the most relevant chunks are retrieved while the irrelevant
ones are ignored, we design a chunk selection algorithm to dynamically select
chunks based on the decreasing speed of the relevance score, leading to a more
relevant selection. Third, to further ensure the precision of the retrieved
chunks, we propose letting LLMs assess whether retrieved chunks are excessive
or lacking and then adjust the amount of context accordingly. Experiments show
that SAGE outperforms baselines by 61.25% in the quality of QA on average.
Moreover, by avoiding retrieving noisy context, SAGE lowers the cost of the
tokens consumed in LLM inference and achieves a 49.41% enhancement in cost
efficiency on average. Additionally, our work offers valuable insights for
boosting RAG.
SAGE, a retrieval-augmented generation framework, improves question-answering tasks by semantically segmenting the corpus and dynamically selecting relevant chunks, enhancing both QA quality and cost efficiency.
retrieval-augmented generation (RAG) Large Language Models (LLMs) semantic segmentation chunk selection relevance score context retrieval cost efficiency