hey @Constantin, i think you might be missing a few preprocessing steps for token classification (i’m assuming that you’re doing something like named entity recognition).

if your input examples have already been split into words then add the is_split_into_words=True argument to the tokenizer
align the labels and tokens - see the tokenize_and_align_labels function in this tutorial: https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/token_classification.ipynb#scrollTo=jmNN-iX1KUHd 4

i’m not familiar with the tensorflow api, but you might also need to specify the DataCollatorForTokenClassification collator for this task