I think in almost all papers they are using pretrained models (2 years ago when i was into nlp that was the case )

Would it hurt to load a larger pretrained model which contains tokens that are not part of my actual data? Is this bad practice, in any way?


It shouldnt word2vec is learning representation of words and the words that are semantically similar are mapped close to each other so even if u use only part of them u just loss some memory/disk to storage word2vec usless for you variable
Also some papers that used pretrained models to emmbeding
https://arxiv.org/pdf/1609.02745.pdf 5