Get Scores for each NE Label
I'm running the run_ner.py script to use the bert-base-german-cased transformer model in the token classification task to train it on custom NE labels and make predictions for German documents. I have 11 labels in total.

I wondered if there is any way to get prediction results (meaning loss, accuracy, precision, etc.) not only for the whole task, but for each label individually. This would make it easier to compare the real perfomance of the model. So that each label has results like:

```
eval_loss = 0.07476427406072617
eval_accuracy_score = 0.9818217086485438
eval_precision = 0.6756756756756757
eval_recall = 0.676378772112383
eval_f1 = 0.6760270410816434
```

Reason I'm asking: the O-Labels (the only thing I kept from BIO tagging) are of course the majority of all labels, therefore the accuracy is quite high as the model correctly predicts most of them, but as a consequence the scores for the real labels get lost in the statistical noise. 

Is there any way to achieve this? 
Thanks in advance.

Maybe @sgugger or @jplu have an idea<|||||>`seqeval` can return you more information, for instance the version in the Datasets library returns all the metrics per label IIRC. The code is [here](https://github.com/huggingface/datasets/blob/8005fed0887236804a07bfdc7dc69298e15dac7c/metrics/seqeval/seqeval.py#L96), you just need to adapt what's inside the `compute_metrics` function to fit your needs.<|||||>Hi @sgugger - that did the trick, thank you!