title: SAGE: A Framework of Precise Retrieval for RAG
summary: Retrieval-augmented generation (RAG) has demonstrated significant proficiency
  in conducting question-answering (QA) tasks within a specified corpus.
  Nonetheless, numerous failure instances of RAG in QA still exist. These
  failures are not solely attributable to the limitations of Large Language
  Models (LLMs); instead, they predominantly arise from the retrieval of
  inaccurate information for LLMs due to two limitations: (1) Current RAG methods
  segment the corpus without considering semantics, making it difficult to find
  relevant context due to impaired correlation between questions and the
  segments. (2) There is a trade-off between missing essential context with fewer
  context retrieved and getting irrelevant context with more context retrieved.
    In this paper, we introduce a RAG framework (SAGE), to overcome these
  limitations. First, to address the segmentation issue without considering
  semantics, we propose to train a semantic segmentation model. This model is
  trained to segment the corpus into semantically complete chunks. Second, to
  ensure that only the most relevant chunks are retrieved while the irrelevant
  ones are ignored, we design a chunk selection algorithm to dynamically select
  chunks based on the decreasing speed of the relevance score, leading to a more
  relevant selection. Third, to further ensure the precision of the retrieved
  chunks, we propose letting LLMs assess whether retrieved chunks are excessive
  or lacking and then adjust the amount of context accordingly. Experiments show
  that SAGE outperforms baselines by 61.25% in the quality of QA on average.
  Moreover, by avoiding retrieving noisy context, SAGE lowers the cost of the
  tokens consumed in LLM inference and achieves a 49.41% enhancement in cost
  efficiency on average. Additionally, our work offers valuable insights for
  boosting RAG.
paper_authors._id: 67c75e18cb29e2a4b0eb0293
paper_authors.name: Jintao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-07T09:26:50.367Z
paper_authors._id: 66c0a08bac74db25de8427ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
paper_authors.fullname: Jintao Zhang
paper_authors.type: user
paper_authors.user: jt-zhang
paper_authors._id: 67c75e18cb29e2a4b0eb0294
paper_authors.name: Guoliang Li
paper_authors._id: 67c75e18cb29e2a4b0eb0295
paper_authors.name: Jinyang Su
paper_title: SAGE: A Framework of Precise Retrieval for RAG
paper_summary: Retrieval-augmented generation (RAG) has demonstrated significant proficiency
  in conducting question-answering (QA) tasks within a specified corpus.
  Nonetheless, numerous failure instances of RAG in QA still exist. These
  failures are not solely attributable to the limitations of Large Language
  Models (LLMs); instead, they predominantly arise from the retrieval of
  inaccurate information for LLMs due to two limitations: (1) Current RAG methods
  segment the corpus without considering semantics, making it difficult to find
  relevant context due to impaired correlation between questions and the
  segments. (2) There is a trade-off between missing essential context with fewer
  context retrieved and getting irrelevant context with more context retrieved.
    In this paper, we introduce a RAG framework (SAGE), to overcome these
  limitations. First, to address the segmentation issue without considering
  semantics, we propose to train a semantic segmentation model. This model is
  trained to segment the corpus into semantically complete chunks. Second, to
  ensure that only the most relevant chunks are retrieved while the irrelevant
  ones are ignored, we design a chunk selection algorithm to dynamically select
  chunks based on the decreasing speed of the relevance score, leading to a more
  relevant selection. Third, to further ensure the precision of the retrieved
  chunks, we propose letting LLMs assess whether retrieved chunks are excessive
  or lacking and then adjust the amount of context accordingly. Experiments show
  that SAGE outperforms baselines by 61.25% in the quality of QA on average.
  Moreover, by avoiding retrieving noisy context, SAGE lowers the cost of the
  tokens consumed in LLM inference and achieves a 49.41% enhancement in cost
  efficiency on average. Additionally, our work offers valuable insights for
  boosting RAG.
paper_ai_summary: SAGE, a retrieval-augmented generation framework, improves question-answering tasks by semantically segmenting the corpus and dynamically selecting relevant chunks, enhancing both QA quality and cost efficiency.
paper_ai_keywords: retrieval-augmented generation (RAG) Large Language Models (LLMs) semantic segmentation chunk selection relevance score context retrieval cost efficiency