When RAG was presented, it did so along this very nice post:

  
      
      ai.facebook.com
  
  
    

Retrieval Augmented Generation: Streamlining the creation of intelligent... 27

Teaching computers to understand how humans write and speak, known as natural language processing or NLP, is one of the oldest challenges in AI research. There has been…


  
  
    
    
  
  


I was wondering if there is a way to obtain the same information shown in the graphs when using HF RAG implementation. That is, the documents weights, as well as the Word-level contribution as referred in the article, or the RAG-Token document posterior as in the paper.
I am aware the document weights can be obtained when doing a forward pass, however these are not obtainable when using the generate() method, which I think would be a “nice have”. I guess now they can be obtained with an extra forward pass before generating, or just tweaking the generate method locally to return them.
However I am not sure how they obtain the posterior for each document. I am guessing it has to do with an average value from the tokens coming from each of the documents (so one would need to “split” the last hidden layer into the document chunks?). Does anyone know better how these could be obtained at generation in order to obtain similar figures as in the article?
Thanks