"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained."
#123
by
dwiraamadhan
- opened
I have successfully transcribed audio to text using the small-whisper model and the message “Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.” appears during the process. Does anyone know what this message means? Does this model automatically store all the tokens input from the user? If yes, are the tokens stored locally on the machine? I would really appreciate it if someone could help me answer this question.