As suggested by @Julio_Marco_A_Silva, best way would be to train on custom data set. If we still face OOV, one way to initialize OOV is using unk_init = torch.Tensor.normal_, while loading pre trained vectors. so that, pytorch will initialize unknown words via Gaussian distribution and this can be applied to train and test sets