Thanks, this is what I was confused about and makes sense now!
So in the case of words which are present in my dataset, but not in the GloVe embeddings, what is standard?

fill all of these as <unk> (randomly initialised, but all the same)
fill each of these with a different random vector
fill each of these with a different random vector and somehow allow for them to be trained

If (3), would setting require_grad=True for all the rows in the embedding layer weights which correspond to these  work? Or is there another way you’d recommend?