It seems that `RagSequenceForGeneration.generate` is computing inaccurate loss value
Hello,

I think `generate` function of `RagSequenceForGeneration` does not seem to compute loss value properly.
As mentioned in the original [RAG paper](https://arxiv.org/pdf/2005.11401.pdf), thorough decoding of Rag-Sequence model needs 2 forward passes.

1. Find candidate sequences through beam search for each document
2. Compute loss for each candidate sequence with another forward pass

Then, return the sequence with the highest likelihood.

I found that `n_docs` argument used in beam search does not passed when `forward` function called for loss computation.

From `RagSequenceForGeneration.generate`,
```
    @torch.no_grad()
    def generate(
        self,
        input_ids: Optional[torch.LongTensor] = None,
        attention_mask: Optional[torch.LongTensor] = None,
        context_input_ids=None,
        context_attention_mask=None,
        doc_scores=None,
        do_deduplication=None,  # defaults to True
        num_return_sequences=None,  # defaults to 1
        num_beams=None,  # defaults to 1
        n_docs=None,
        **model_kwargs
    ):
            ...

            # then, run model forwards to get nll scores:
            if input_ids is not None:
                new_input_ids = input_ids[index : index + 1].repeat(num_candidates, 1)
                outputs = self(new_input_ids, labels=output_sequences, exclude_bos_score=True)
            ...
```

I think same documents used for beam search, also need to be used for loss computation.
If `n_docs` other than default value is passed to the `generate` function, it will end up computing loss somewhat inaccurate, possibly can change the final generated sequence.
If this is the case, I think `generate` function should be fixed, which can be simply done by just passing `n_docs` argument to `forward` inside `generate`.

Please correct me if there are any mistakes.
Thanks

@patrickvonplaten 
Maybe of interest to @lhoestq too<|||||>Looking into it next week - sorry for being so late here<|||||>Hey @repun,

I'm not 100% following everything here. Could you maybe open a PR so that we can look into it with some code changes? 
In general, I think it's totally fine if the number of documents to retrieve during inference in `generate()` differs from the number of documents to retrieve during training. Could you explain in a bit more detail why the loss value is not accurate and how the loss value has to do with `generate()` or are we just talking about inference (and not training at all here?)<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.