title: Similarity is Not All You Need: Endowing Retrieval Augmented Generation
    with Multi Layered Thoughts
summary: In recent years, large language models (LLMs) have made remarkable
  achievements in various domains. However, the untimeliness and cost of
  knowledge updates coupled with hallucination issues of LLMs have curtailed
  their applications in knowledge intensive tasks, where retrieval augmented
  generation (RAG) can be of help. Nevertheless, existing retrieval augmented
  models typically use similarity as a bridge between queries and documents and
  follow a retrieve then read procedure. In this work, we argue that similarity
  is not always the panacea and totally relying on similarity would sometimes
  degrade the performance of retrieval augmented generation. To this end, we
  propose MetRag, a Multi layEred Thoughts enhanced Retrieval Augmented
  Generation framework. To begin with, beyond existing similarity oriented
  thought, we embrace a small scale utility model that draws supervision from an
  LLM for utility oriented thought and further come up with a smarter model by
  comprehensively combining the similarity and utility oriented thoughts.
  Furthermore, given the fact that the retrieved document set tends to be huge
  and using them in isolation makes it difficult to capture the commonalities and
  characteristics among them, we propose to make an LLM as a task adaptive
  summarizer to endow retrieval augmented generation with compactness-oriented
  thought. Finally, with multi layered thoughts from the precedent stages, an LLM
  is called for knowledge augmented generation. Extensive experiments on
  knowledge-intensive tasks have demonstrated the superiority of MetRag.
paper_authors._id: 6659496e6827c52ec8667980
paper_authors.name: Chunjing Gan
paper_authors._id: 6659496e6827c52ec8667981
paper_authors.name: Dan Yang
paper_authors._id: 6659496e6827c52ec8667982
paper_authors.name: Binbin Hu
paper_authors._id: 6659496e6827c52ec8667983
paper_authors.name: Hanxiao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:12:36.859Z
paper_authors._id: 643659b5ad9b9147de282fd9
paper_authors.avatarUrl: /avatars/316e76c611c5700cdc70bef8eab4995a.svg
paper_authors.fullname: HanXiao Zhang
paper_authors.type: user
paper_authors.user: nighttrain
paper_authors._id: 6659496e6827c52ec8667984
paper_authors.name: Siyuan Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:19:50.551Z
paper_authors._id: 640f7083208821a59b74c757
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678735253848-640f7083208821a59b74c757.jpeg
paper_authors.fullname: Siyuan Li
paper_authors.type: user
paper_authors.user: Lupin1998
paper_authors._id: 6659496e6827c52ec8667985
paper_authors.name: Ziqi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:13:49.472Z
paper_authors._id: 64264f4885f26ab94af3e1c3
paper_authors.avatarUrl: /avatars/db60c953e3d0c9befb1ff96815320ccb.svg
paper_authors.fullname: Ziqi Liu
paper_authors.type: user
paper_authors.user: ziqiliu
paper_authors._id: 6659496e6827c52ec8667986
paper_authors.name: Yue Shen
paper_authors._id: 6659496e6827c52ec8667987
paper_authors.name: Lin Ju
paper_authors._id: 6659496e6827c52ec8667988
paper_authors.name: Zhiqiang Zhang
paper_authors._id: 6659496e6827c52ec8667989
paper_authors.name: Jinjie Gu
paper_authors._id: 6659496e6827c52ec866798a
paper_authors.name: Lei Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:14:47.296Z
paper_authors._id: 64588a9caa426bae5e080567
paper_authors.avatarUrl: /avatars/76c11bfbe64058014d4b2b89d18ba2e4.svg
paper_authors.fullname: Lei Liang
paper_authors.type: user
paper_authors.user: square0083
paper_authors._id: 6659496e6827c52ec866798b
paper_authors.name: Jun Zhou
paper_title: Similarity is Not All You Need: Endowing Retrieval Augmented Generation
    with Multi Layered Thoughts
paper_summary: In recent years, large language models (LLMs) have made remarkable
  achievements in various domains. However, the untimeliness and cost of
  knowledge updates coupled with hallucination issues of LLMs have curtailed
  their applications in knowledge intensive tasks, where retrieval augmented
  generation (RAG) can be of help. Nevertheless, existing retrieval augmented
  models typically use similarity as a bridge between queries and documents and
  follow a retrieve then read procedure. In this work, we argue that similarity
  is not always the panacea and totally relying on similarity would sometimes
  degrade the performance of retrieval augmented generation. To this end, we
  propose MetRag, a Multi layEred Thoughts enhanced Retrieval Augmented
  Generation framework. To begin with, beyond existing similarity oriented
  thought, we embrace a small scale utility model that draws supervision from an
  LLM for utility oriented thought and further come up with a smarter model by
  comprehensively combining the similarity and utility oriented thoughts.
  Furthermore, given the fact that the retrieved document set tends to be huge
  and using them in isolation makes it difficult to capture the commonalities and
  characteristics among them, we propose to make an LLM as a task adaptive
  summarizer to endow retrieval augmented generation with compactness-oriented
  thought. Finally, with multi layered thoughts from the precedent stages, an LLM
  is called for knowledge augmented generation. Extensive experiments on
  knowledge-intensive tasks have demonstrated the superiority of MetRag.
paper_ai_keywords: large language models (LLMs) retrieval augmented generation (RAG) similarity retrieve then read multi-layered thoughts utility model utility oriented thought task adaptive summarizer compactness-oriented thought knowledge augmented generation