Solved by nielsr in post #2 
              
              
                What is typically done is, you tokenize each word of the annotated dataset you have, check how many tokens it has been tokenized into, and then either only label the first wordpiece token, or label them all. 
Small example: 
Suppose you have the sentence “hello my name is Niels”, and the CoNLL datas…