Answers.Body: <p>LLM Embedding has nothing to do with RAG's embedding.</p>
  <p>Input --&gt; RAG --&gt; RAG's Embedding --&gt; Search similar RAG's embedded documents --&gt; Get those embedded documents --&gt; Decode it --&gt; Now we have input and documents in text format --&gt; Give text format input and document to LLM.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-08T08:54:10.830Z
Answers.LastActivityDate: 2024-06-08T08:54:10.830Z
Answers.UserDisplayName: Aamod Thakur
Answers.Body: <p>In RAG systems, document embeddings used for semantic search are separate from the LLM's internal embeddings. The RAG system tokenizes and vectorizes the user's query using its own embedding model to find relevant documents in the vector database. Only the text content of the retrieved documents is returned to the LLM, which then processes it using its own internal embedding model to incorporate it into the context for generating a response.</p>
  <p>Some sites claim that the RAG performance can be improved by using a shared embeddings model between the LLM and RAG.  However, this only applies to the prompt which would not have to be converted to embeddings twice.  I suppose this could become very significant for large prompts.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-08T22:22:42.197Z
Answers.LastActivityDate: 2024-06-08T22:22:42.197Z
Answers.UserDisplayName: Mark_Sagecy
Body: <p>I have read and seen that you can use a different embeddings model for RAG embedding vectors than was used to train the original RAG.</p>
  <p>I can't get my head around how that can work!</p>
  <p>How is it possible to do semantic matching of the input prompt against RAG if the LLM is using a different embeddings model than the RAG embeddings?</p>
  
Tags: <embeddings><vector-semantics>
Title: How is it possible that RAG embeddings implemented with a different embedding model than the pre-trained LLM can work?
UserDisplayName: Mark_Sagecy
ThreadText: # How is it possible that RAG embeddings implemented with a different embedding model than the pre-trained LLM can work?
  
  (Asked by: username_0 on 2024-06-07)
  
  I have read and seen that you can use a different embeddings model for RAG embedding vectors than was used to train the original RAG.
  
  I can't get my head around how that can work!
  
  How is it possible to do semantic matching of the input prompt against RAG if the LLM is using a different embeddings model than the RAG embeddings?
  
  **Tags:** embeddings, vector-semantics
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-06-08. Score: 1
  
  LLM Embedding has nothing to do with RAG's embedding.
  
  Input --> RAG --> RAG's Embedding --> Search similar RAG's embedded documents --> Get those embedded documents --> Decode it --> Now we have input and documents in text format --> Give text format input and document to LLM.
  
  
  ---
  
  ### Answer by username_0 on 2024-06-08. Score: 0
  
  In RAG systems, document embeddings used for semantic search are separate from the LLM's internal embeddings. The RAG system tokenizes and vectorizes the user's query using its own embedding model to find relevant documents in the vector database. Only the text content of the retrieved documents is returned to the LLM, which then processes it using its own internal embedding model to incorporate it into the context for generating a response.
  
  Some sites claim that the RAG performance can be improved by using a shared embeddings model between the LLM and RAG. However, this only applies to the prompt which would not have to be converted to embeddings twice. I suppose this could become very significant for large prompts.
  
  
  ---
  
  