you can use torch.cat to concatenate two tensors (the one-hot vector with the output of nn.Embedding layer).
Something like below:
import torch.nn as nn

num_sentences = 4
num_unique_words = 20
indices = torch.tensor([1,1,2,0,0,0,1])
one_hot_buffer = torch.eye(num_sentences)
embeddings = nn.Embedding(num_unique_words, 5)

one_hot = one_hot_buffer[indices]
embed = embeddings(indices)
concatenated = torch.cat((embed, one_hot), dim=1)