A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
Retrieval Augmented Generation (RAG) represents a significant advancement in
artificial intelligence combining a retrieval phase with a generative phase,
with the latter typically being powered by large language models (LLMs). The
current common practices in RAG involve using "instructed" LLMs, which are
fine-tuned with supervised training to enhance their ability to follow
instructions and are aligned with human preferences using state-of-the-art
techniques. Contrary to popular belief, our study demonstrates that base models
outperform their instructed counterparts in RAG tasks by 20% on average under
our experimental settings. This finding challenges the prevailing assumptions
about the superiority of instructed LLMs in RAG applications. Further
investigations reveal a more nuanced situation, questioning fundamental aspects
of RAG and suggesting the need for broader discussions on the topic; or, as
Fromm would have it, "Seldom is a glance at the statistics enough to understand
the meaning of the figures".
6679223125771fc36d9ac38b
Florin Cuconasu
claimed_verified
2024-06-24T13:10:24.561Z
62bfff6788fdef8ecde8c45b
/avatars/637da88b8dce01bdc2a4c1319cc882e3.svg
Florin Cuconasu
user
florin-hf
6679223125771fc36d9ac38c
Giovanni Trappolini
admin_assigned
2024-06-24T14:02:18.536Z
631e51736d6a5870f3d7d0fe
/avatars/9eb93a278d01c3abd6dc28c708f0d107.svg
Giovanni Trappolini
user
rarinpear
6679223125771fc36d9ac38d
Nicola Tonellotto
admin_assigned
2024-06-24T14:02:23.972Z
66505657f329f5aa4e1e7466
https://cdn-avatars.huggingface.co/v1/production/uploads/66505657f329f5aa4e1e7466/ierINXTTY_FNY69aBBtyI.webp
Nicola Tonellotto
user
ntonellotto
6679223125771fc36d9ac38e
Fabrizio Silvestri
admin_assigned
2024-06-24T14:02:29.026Z
6075d282134c000d1ae10d54
/avatars/8691622e1f6325bc378b2268c119b9a0.svg
Fabrizio Silvestri
user
fabrizio-silvestri
A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
Retrieval Augmented Generation (RAG) represents a significant advancement in
artificial intelligence combining a retrieval phase with a generative phase,
with the latter typically being powered by large language models (LLMs). The
current common practices in RAG involve using "instructed" LLMs, which are
fine-tuned with supervised training to enhance their ability to follow
instructions and are aligned with human preferences using state-of-the-art
techniques. Contrary to popular belief, our study demonstrates that base models
outperform their instructed counterparts in RAG tasks by 20% on average under
our experimental settings. This finding challenges the prevailing assumptions
about the superiority of instructed LLMs in RAG applications. Further
investigations reveal a more nuanced situation, questioning fundamental aspects
of RAG and suggesting the need for broader discussions on the topic; or, as
Fromm would have it, "Seldom is a glance at the statistics enough to understand
the meaning of the figures".
Retrieval Augmented Generation (RAG) tasks are better performed by base models than instructed LLMs, challenging assumptions about the superiority of fine-tuned LLMs.
Retrieval Augmented Generation (RAG) large language models (LLMs) instructed LLMs fine-tuning human preferences state-of-the-art techniques