Hey @mawilson if you want to add new tokens to the vocabulary, then in general you’ll need to resize the embedding layers with
model.resize_token_embeddings(len(tokenizer))

You can see a full example in the docs 4 - does that help solve your problem?