Bart-in-RAG and Yacine’s Bart-in-ELI5 is very similar in design.
Overall pipeline, RAG and Bart-ELI5 query ‘question’ to retrieve passages (5 passages as default), then concat both question-passage together as input for Bart-encoder.
In the paper and Huggingface’s pretrained weights, RAG uses NQ dataset, so it’s short-form QA unlike ELI5.
Note that from 1 example we have in NQ, in RAG, we will extend it to 5 examples :
Consider one training data of original NQ : Q / A (Question and Answer pair)
In RAG, this NQ training data becomes : Q-P1 / A , Q-P2 / A , Q-P3 / A , Q-P4 / A, Q-P5 / A
where P1-P5 are retrieve passages. And we train Bart with these new pairs.