What is typically done is, you tokenize each word of the annotated dataset you have, check how many tokens it has been tokenized into, and then either only label the first wordpiece token, or label them all.
Small example:
Suppose you have the sentence “hello my name is Niels”, and the CoNLL dataset has this labeled as:
hello O
my O
name O
is O
niels B-PER

Then what we do is the following:

option 1, label all tokens of a word (i.e. propagate the label on all tokens)

from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

words = ["hello", "my", "name", "is", "niels"]
word_labels = ["O", "O", "O", "O", "B-PER"]
# convert word labels to integer labels
label2id  = {"O": 0, "B-PER":1}
word_labels = [label2id[label] for label in word_labels]

tokens = []
labels = []
for word, label in zip(words, word_labels):
     # tokenize the word
     word_tokens = tokenizer.tokenize(word)
     # propagate label to all tokens of the given word
     tokens.extend(word_tokens)
     labels.extend([label] * len(word_tokens))


option 2, only label the first token of a word, set labels of all remaining tokens to -100

words = ["hello", "my", "name", "is", "niels"]
word_labels = ["O", "O", "O", "O", "B-PER"]
# convert word labels to integer labels
label2id  = {"O": 0, "B-PER":1}
word_labels = [label2id[label] for label in word_labels]

tokens = []
labels = []
for word, label in zip(words, word_labels):
     # tokenize the word
     word_tokens = tokenizer.tokenize(word)
     tokens.extend(word_tokens)
     # only label the first wordpiece
     labels.extend([label] + [-100] * (len(word_tokens) - 1))

The reason we set the remaining tokens to -100 is because this is the ignore_index of PyTorch’ CrossEntropyLoss 1. This means that those labels will not be taken into account by the loss function, hence no gradients will be computed for those.
Which of the 2 options you choose is a design choice, mainly. In practice, both perform well. I’ve made 2 notebooks (one for each of both options), in my repo here 2.