I am looking for a way to handle unknown words on the fly in the encoder side. I want to freeze pre-trained word embeddings during training. And for words that are not in pre-trained embeddings I would like to generate custom word embeddings based on some logic (char n-gram, morphology etc.,).
embedding = nn.Embedding(num_embedding, embedding_dim)
embedding.weight = nn.Parameter(pretrained_embedding)
embedding.weight.requires_grad = False

In the forward function, while I get embedded = self.embedding(word_inputs) is there a way to do get custom representation for unknown words in the embedded tensor? Since the embedding matrix dimension is fixed  I am not sure how to do this?
Any help is much appreciated.