Return output probabilities with Generate function 
# 🚀 Feature request

Output the token probabilities along with the tokens when generating sequence.

## Motivation

For understanding model confidence, this is quite useful. 
Also, for abstractive QA with long contexts, one needs to use doc-strides to take into account the contexts & then choose the best answer according to the probability of the generated text.


## Your contribution

I can try submitting a PR for non-beam decoding, but guidance would be appreciated.
Also, are there any existing solutions to this issue? If so, what & where?
Duplicate of https://github.com/huggingface/transformers/issues/7654 . But yes, it seems like many people are asking for this feature and it should be quite straight-forward to implement it... -> feel free to give a shot :-) <|||||>Hi. When can we expect this to be released? 😊<|||||>Next week Tuesday or Wednesday :-) <|||||>Hey! Can anyone point me to the API/code location of this feature? Sorry if I have missed something. :) Thank you!<|||||>@moqingyan https://huggingface.co/transformers/internal/generation_utils.html<|||||>> @moqingyan https://huggingface.co/transformers/internal/generation_utils.html

Thank you!

My problem was: I set the configuration `output_scores` to `True` in the `generate` function and failed obtain the scores in the returned results. After I struggled in the libraries for hours, I finally figured out I also need to set `return_dict_in_generate` to `True` to obtain the attention values from the `generate` function. :) 

I think this behavior is non-intuitive, as I have specified I need the scores in the output already. But anyway I figured this out. Hope this comment is helpful to anyone who runs into this issue. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.