Allow one to return encoder attentions in seq2seq generation
# 🚀 Feature request

Please could we have the ability to return attention weights from the decoded generated tokens to the encoded source?

## Motivation

To attribute the decoded text. E.g in the summarization task we want to see where the decoder was paying attention to in the source.

## Your contribution

May be able to look into a PR but stretched for time at the minute.

FairSeq has implemented this capability I believe.
Hi @aced125, I agree that this functionality should be provided :-) 

I think in the PR, one has to include a `output_attention` argument to the `generate()` function and then make sure that the output idx are correct! 
Before starting this PR, this PR should probably be solved before: https://github.com/huggingface/transformers/issues/3880<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>want to take a look at this soon<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>#6735 is a first step to allow for this feature<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
