title: From RAGs to rich parameters: Probing how language models utilize
    external knowledge over parametric information for factual queries
summary: Retrieval Augmented Generation (RAG) enriches the ability of language models
  to reason using external context to augment responses for a given user prompt.
  This approach has risen in popularity due to practical applications in various
  applications of language models in search, question/answering, and chat-bots.
  However, the exact nature of how this approach works isn't clearly understood.
  In this paper, we mechanistically examine the RAG pipeline to highlight that
  language models take shortcut and have a strong bias towards utilizing only the
  context information to answer the question, while relying minimally on their
  parametric memory. We probe this mechanistic behavior in language models with:
  (i) Causal Mediation Analysis to show that the parametric memory is minimally
  utilized when answering a question and (ii) Attention Contributions and
  Knockouts to show that the last token residual stream do not get enriched from
  the subject token in the question, but gets enriched from other informative
  tokens in the context. We find this pronounced shortcut behaviour true across
  both LLaMa and Phi family of models.
paper_authors._id: 6672553486f6b3d126834783
paper_authors.name: Hitesh Wadhwa
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-19T07:37:38.560Z
paper_authors._id: 64ebf8dff827684f65658073
paper_authors.avatarUrl: /avatars/60cf2037ad7e1e2adb135660bd28204f.svg
paper_authors.fullname: Hitesh Wadhwa
paper_authors.type: user
paper_authors.user: wadhwahitesh
paper_authors._id: 6672553486f6b3d126834784
paper_authors.name: Rahul Seetharaman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:40:07.803Z
paper_authors._id: 654295d34f89f533be6d4d2b
paper_authors.avatarUrl: /avatars/4ac23b7ef93848232b46896add4f19a2.svg
paper_authors.fullname: Rahul Seetharaman
paper_authors.type: user
paper_authors.user: rahulseetharaman
paper_authors._id: 6672553486f6b3d126834785
paper_authors.name: Somyaa Aggarwal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:40:13.790Z
paper_authors._id: 65df214be98700500d4cb0f4
paper_authors.avatarUrl: /avatars/e370887a53c8740944b17c65a4f5eb07.svg
paper_authors.fullname: Somyaa Aggarwal
paper_authors.type: user
paper_authors.user: Somyaa27
paper_authors._id: 6672553486f6b3d126834786
paper_authors.name: Reshmi Ghosh
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-19T12:49:23.836Z
paper_authors._id: 63228cad95d6f717a8c8e1ec
paper_authors.avatarUrl: /avatars/5fa606f63f841879af1a7617366d18b4.svg
paper_authors.fullname: Reshmi Ghosh
paper_authors.type: user
paper_authors.user: reshmighosh
paper_authors._id: 6672553486f6b3d126834787
paper_authors.name: Samyadeep Basu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:40:19.705Z
paper_authors._id: 6384d891cafe4e4ac4060206
paper_authors.avatarUrl: /avatars/45eb04bf6e0ed1a4daa14070411b693f.svg
paper_authors.fullname: Samyadeep
paper_authors.type: user
paper_authors.user: samyadeepbasu
paper_authors._id: 6672553486f6b3d126834788
paper_authors.name: Soundararajan Srinivasan
paper_authors._id: 6672553486f6b3d126834789
paper_authors.name: Wenlong Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:40:29.254Z
paper_authors._id: 64b9e76820c3226970ab4abf
paper_authors.avatarUrl: /avatars/1b361d83120f18f7cb950ba039991f6a.svg
paper_authors.fullname: WENLONG ZHAO
paper_authors.type: user
paper_authors.user: Warren7771818
paper_authors._id: 6672553486f6b3d12683478a
paper_authors.name: Shreyas Chaudhari
paper_authors._id: 6672553486f6b3d12683478b
paper_authors.name: Ehsan Aghazadeh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:40:38.293Z
paper_authors._id: 604f78723050a33ebb17ef63
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1615820858280-noauth.jpeg
paper_authors.fullname: Ehsan Aghazadeh
paper_authors.type: user
paper_authors.user: EhsanAghazadeh
paper_title: From RAGs to rich parameters: Probing how language models utilize
    external knowledge over parametric information for factual queries
paper_summary: Retrieval Augmented Generation (RAG) enriches the ability of language models
  to reason using external context to augment responses for a given user prompt.
  This approach has risen in popularity due to practical applications in various
  applications of language models in search, question/answering, and chat-bots.
  However, the exact nature of how this approach works isn't clearly understood.
  In this paper, we mechanistically examine the RAG pipeline to highlight that
  language models take shortcut and have a strong bias towards utilizing only the
  context information to answer the question, while relying minimally on their
  parametric memory. We probe this mechanistic behavior in language models with:
  (i) Causal Mediation Analysis to show that the parametric memory is minimally
  utilized when answering a question and (ii) Attention Contributions and
  Knockouts to show that the last token residual stream do not get enriched from
  the subject token in the question, but gets enriched from other informative
  tokens in the context. We find this pronounced shortcut behaviour true across
  both LLaMa and Phi family of models.
paper_ai_summary: The study investigates the RAG framework and reveals that language models prefer using external context over their internal parametric memory, as demonstrated through causal mediation analysis and attention contributions.
paper_ai_keywords: Retrieval Augmented Generation RAG language models parametric memory causal mediation analysis attention contributions LLaMa Phi