question about `add_special_tokens` and embedding layer
# ❓ Questions & Help

when I call add_special_tokens in the tokniezer, new token that is not in the existing vocab is added. At this time, the size is different from the existing embedding layer, what happens inside?

1. Abandon the existing embedding layer and train a new.
2. train only new tokens (expand existing embedding layer)
When you add special tokens to the tokenizer, you should resize the model embedding matrix so that it contains the extra embeddings.

You should then do an additional training/fine-tuning on data containing those special tokens so that the model may tune this new column in the embedding matrix.<|||||>thanks !