Hi,
I just checked my notebooks 2, I actually do evaluate at the named entity level, rather than at the token level. I do this using the seqeval library.
It’s actually the classification_report that shows the performance for the different NER categories.