Answers.Body: <p>In the first equation, you can see that in the sequence likelihood (the big product) <span class="math-container">$z$</span> is constant, which means that you first find which are the interesting documents, than you give your LLM these document, and you ask it to answer (for the whole generation, these documents are always the same)</p>
  <p>In the second case, you can see that for every step (<span class="math-container">$i$</span>), you have a corresponding <span class="math-container">$z_i$</span> (a set of documents) that is used in the summation. However, you can see that that summation is just an expectation for the <span class="math-container">$y_i$</span> token. This means that, if earlier you first find K documents and then generate an answer, now you have to find interesting documents every time you have to sample a token.</p>
  <p>In other words:</p>
  <ul>
  <li>in the first case: (1) sample K documents based on query (2) ask the LLM to generate the first token (3) concat the new token to the generated sentence (4) repeat 2 until EOS token is chosen</li>
  <li>in the second case: (1) find top K documents based on the current generation&amp;query (2) feed the documents to the LLM, (3) generate next token (4) concat the new token to the current generated sequence, (5) repeat 1 until EOS is sampled</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-28T19:58:14.247Z
Answers.LastActivityDate: 2024-04-28T19:58:14.247Z
Answers.UserDisplayName: Alberto
Body: <p>When I start to learn about RAG, I can't understand the difference between the RAG-Sequence Model and RAG-Token Model.</p>
  <p>First of all, I see that for <strong>RAG-Sequence Model</strong> we use just one document to generate the hole output and for <strong>RAG-Token Model</strong> for each token we use all the k document chosen.</p>
  <p>But, when I see the equation in the <a href="https://arxiv.org/pdf/2005.11401" rel="nofollow noreferrer">paper</a>, I can't understand why they make a summation in the case of RAG-Sequence Model while we generate the output from one document.</p>
  <p><a href="https://i.sstatic.net/iVGh6eJj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/iVGh6eJj.png" alt="enter image description here" /></a></p>
  
Tags: <papers><information-retrieval>
Title: What is the difference between RAG-Sequence Model and RAG-Token Model?
UserDisplayName: LAILA EL OUEDEGHYRY
ThreadText: # What is the difference between RAG-Sequence Model and RAG-Token Model?
  
  (Asked by: username_0 on 2024-04-28)
  
  When I start to learn about RAG, I can't understand the difference between the RAG-Sequence Model and RAG-Token Model.
  
  First of all, I see that for **RAG-Sequence Model** we use just one document to generate the hole output and for **RAG-Token Model** for each token we use all the k document chosen.
  
  But, when I see the equation in the [paper](https://arxiv.org/pdf/2005.11401), I can't understand why they make a summation in the case of RAG-Sequence Model while we generate the output from one document.
  
  [![enter image description here](https://i.sstatic.net/iVGh6eJj.png)](https://i.sstatic.net/iVGh6eJj.png)
  
  **Tags:** papers, information-retrieval
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-28. Score: 1
  
  In the first equation, you can see that in the sequence likelihood (the big product) $z$ is constant, which means that you first find which are the interesting documents, than you give your LLM these document, and you ask it to answer (for the whole generation, these documents are always the same)
  
  In the second case, you can see that for every step ($i$), you have a corresponding $z\_i$ (a set of documents) that is used in the summation. However, you can see that that summation is just an expectation for the $y\_i$ token. This means that, if earlier you first find K documents and then generate an answer, now you have to find interesting documents every time you have to sample a token.
  
  In other words:
  
  * in the first case: (1) sample K documents based on query (2) ask the LLM to generate the first token (3) concat the new token to the generated sentence (4) repeat 2 until EOS token is chosen
  * in the second case: (1) find top K documents based on the current generation&query (2) feed the documents to the LLM, (3) generate next token (4) concat the new token to the current generated sequence, (5) repeat 1 until EOS is sampled
  
  
  ---
  
  