[Feature Request] Extract Predictions from Trainer
# 🚀 Feature request

Add the possibility to return predictions along with its example id in the new Trainer class.

## Motivation

When working with extractive QA (i.e.: SQuAD), you get back the best predictions, but, the current example for running squad uses the old, plain training/eval script, without the new Trainer class.

Additionally, there are other tasks where predictions can be extremely useful (i.e.: Multiple Choice).

Adding such functionality in the Trainer class could solve this and unify both question answering and multiple choice examples.

## Your contribution

I am familiarized with the code (both the Trainer class and the old train/eval script), so I could submit a PR with the new functionality.
Unless I'm mistaken, Question-answering does support Trainer. Is a specific feature missing for your use case?<|||||>Hi @julien-c!

At the time of writing the PR was not merged yet (and I was not aware of it, my bad). It is great!

In any case, the previous example (the one without trainer) outputted both `predictions` and `nbest_predictions`, which are crucial for error analysis.

The new example with trainer looses this ability (because the new dataset/trainer API does not support it). Adding this feature could benefit not only SQuAD, but other datasets too (i.e.: RACE).

What do you think on this regard?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Any updates on this matter? I recently started using `transformers` and running the multiple-choice example I realized there's an output of the overall accuracy but I can't find the predictions.  Currently going through the code to obtain them...<|||||>Hi @dianags, 

Might be a little late, but tired of waiting, I programmed my own solution that output predictions and some more data, the code can be found [here](https://github.com/geblanco/mc_transformers)

Best,<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
