Adding end-to-end retriever training to RAG with RAY implementation. 
# 🚀 Feature request


Use of RAY to run separate processes for retrieve document indexes, training the system, and re-initialize the indexes with an updated context encoder.

## Motivation
Recent [papers](https://arxiv.org/abs/2101.00408) have shown that fine-tuning the entire retriever gives huge gains for QA tasks. Also having able to fine-tune end-end manner can give better results in different domains.


The idea is to train the RAG as is, but we keep updating the gradients of the context encoder with the supervised loss function (doc score mentioned in the RAG). Then in every n-steps, we re-initialize the embedding and indexes with updated context encoder weights. 


REALM [does this with a back-and-forth process ](https://github.com/google-research/language/tree/master/language/realm#running-the-code)which is only run on a single GPU.


As discussed in this [issue](https://github.com/huggingface/transformers/issues/9646#issuecomment-775309123), @lhoestq  suggested it would be easier to complete this with RAY since we can have separate actors. 


@richardliaw
@amogkam 
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.