marrrcin:

This is true, but my understanding of @chrisdoyleIE 's answer is that extending the existing vocabulary still goes into the flow for fine-tuning, as the internal representation of already existing tokens will not change, am I right?


Yeah, internal representations might not change (depends on what is exactly meant by this) but while fine-tuning the model would learn new facts (or relations) about (or between) existing tokens and new ones added as mentioned by @chrisdoyleIE. At least this is what I can think of.
Hope this makes sense.