Hi,
If you’re using a fast tokenizer (such as BertTokenizerFast), you can add use the offsets to know if a token is a special token/the first wordpiece of a word or not. Small example:
from transformers import BertTokenizerFast

tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
text = "hello my name is niels"

encoding = tokenizer(text, return_offsets_mapping=True)

for id, offset in zip(encoding.input_ids, encoding.offset_mapping):
  print(tokenizer.decode([id]), offset)

This returns:
[CLS] (0, 0)
hello (0, 5)
my (6, 8)
name (9, 13)
is (14, 16)
ni (17, 19)
##els (19, 22)
[SEP] (0, 0)

As you can see, the offsets for special tokens are (0, 0), and if offset[0] of a particular token is equal to offset[1] of the previous token, then we know that it’s a subword token that’s not the first one of a word.
You can write this in a (rather long) list comprehension, to filter the predictions:
true_indices = [1] + [idx for idx, offset in enumerate(encoding.offset_mapping) if offset != (0, 0) and offset[0] != encoding.offset_mapping[idx-1][1]]
true_predictions = predictions.numpy()[true_indices]