title: A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
summary: Retrieval Augmented Generation (RAG) represents a significant advancement in
  artificial intelligence combining a retrieval phase with a generative phase,
  with the latter typically being powered by large language models (LLMs). The
  current common practices in RAG involve using "instructed" LLMs, which are
  fine-tuned with supervised training to enhance their ability to follow
  instructions and are aligned with human preferences using state-of-the-art
  techniques. Contrary to popular belief, our study demonstrates that base models
  outperform their instructed counterparts in RAG tasks by 20% on average under
  our experimental settings. This finding challenges the prevailing assumptions
  about the superiority of instructed LLMs in RAG applications. Further
  investigations reveal a more nuanced situation, questioning fundamental aspects
  of RAG and suggesting the need for broader discussions on the topic; or, as
  Fromm would have it, "Seldom is a glance at the statistics enough to understand
  the meaning of the figures".
paper_authors._id: 6679223125771fc36d9ac38b
paper_authors.name: Florin Cuconasu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T13:10:24.561Z
paper_authors._id: 62bfff6788fdef8ecde8c45b
paper_authors.avatarUrl: /avatars/637da88b8dce01bdc2a4c1319cc882e3.svg
paper_authors.fullname: Florin Cuconasu
paper_authors.type: user
paper_authors.user: florin-hf
paper_authors._id: 6679223125771fc36d9ac38c
paper_authors.name: Giovanni Trappolini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T14:02:18.536Z
paper_authors._id: 631e51736d6a5870f3d7d0fe
paper_authors.avatarUrl: /avatars/9eb93a278d01c3abd6dc28c708f0d107.svg
paper_authors.fullname: Giovanni Trappolini
paper_authors.type: user
paper_authors.user: rarinpear
paper_authors._id: 6679223125771fc36d9ac38d
paper_authors.name: Nicola Tonellotto
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T14:02:23.972Z
paper_authors._id: 66505657f329f5aa4e1e7466
paper_authors.avatarUrl: /avatars/f5af1279f6761c2a12cf7fe4781301ad.svg
paper_authors.fullname: Nicola Tonellotto
paper_authors.type: user
paper_authors.user: ntonellotto
paper_authors._id: 6679223125771fc36d9ac38e
paper_authors.name: Fabrizio Silvestri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T14:02:29.026Z
paper_authors._id: 6075d282134c000d1ae10d54
paper_authors.avatarUrl: /avatars/8691622e1f6325bc378b2268c119b9a0.svg
paper_authors.fullname: Fabrizio Silvestri
paper_authors.type: user
paper_authors.user: fabrizio-silvestri
paper_title: A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
paper_summary: Retrieval Augmented Generation (RAG) represents a significant advancement in
  artificial intelligence combining a retrieval phase with a generative phase,
  with the latter typically being powered by large language models (LLMs). The
  current common practices in RAG involve using "instructed" LLMs, which are
  fine-tuned with supervised training to enhance their ability to follow
  instructions and are aligned with human preferences using state-of-the-art
  techniques. Contrary to popular belief, our study demonstrates that base models
  outperform their instructed counterparts in RAG tasks by 20% on average under
  our experimental settings. This finding challenges the prevailing assumptions
  about the superiority of instructed LLMs in RAG applications. Further
  investigations reveal a more nuanced situation, questioning fundamental aspects
  of RAG and suggesting the need for broader discussions on the topic; or, as
  Fromm would have it, "Seldom is a glance at the statistics enough to understand
  the meaning of the figures".
paper_ai_keywords: Retrieval Augmented Generation (RAG) large language models (LLMs) instructed LLMs fine-tuned supervised training human preferences base models instructed counterparts RAG tasks experimental settings