Write inference evaluation
### Feature request

This feature request is strongly inspired by T5X. They write a log every time they do an evaluation. The log is a json-lines-file that is saved in inference_eval/task-1000.jsonl. "task" is the current task, and "1000" is the checkpoint. The file is generic, and looks like this:
```json
{
  "input": {
    "inputs_pretokenized": "Hello World",
    "inputs": [###,###,###],
    "targets_pretokenized": "Hallo verden",
    "targets": [###,###,###]
  },
  "target": "Hallo verden",
  "output": "Hei verden",
  "prediction": "Hei verden"
}
```
Long inputs, like audio and images, will typically be truncated.


### Motivation

This simply makes debugging a lot easier. The jsonl-format makes it really easy to open this in another program for more thorough study. Since the same file is predicted at every step, it makes it really easy to follow the development of a single target.

### Your contribution

I would be glad to make feedback on such an implementation. I am not exactly sure where this feature should be implemented today.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.