This is where you are supposed to use the nn.Embedding layer.
For example:
self.embedding = nn.Embedding(vocabulary_size, embedding_dimensions)

The vocabulary_size is the how many words you have in your vocabulary and the embedding_dimensions is the size of the vector that you are using for each word. (Constructing in essence a vocab x dimensions tensor, which then is used by the embedding layer to construct your final seq_len x batch x dimensions tensor that RNNs need)
and you use it in the forward path before the RNN:
def forward(self, inputs, hidden):
        embeddings = self.embeddings(inputs)
        output, hidden = self.lstm(embeddings, hidden)

You could either initialise the weights of the embedding layer randomly from a distribution:
self.embeddings.weight.data.uniform_(-0.1, 0.1)

Or if you want to re-use trained embeddings from some other source (say word2vec) you must pass the vectors to the embedding layer:
self.embeddings.weight.data.copy_(torch.from_numpy(embedding_weights))

You need to construct an id to vector mapping during your tokenization phase (I don’t know if there is a different/ better way to go about this). Something like this:
from gensim.models import Word2Vec

embeddings = Word2Vec.load('word2vec.model')

vectors = {}
wordToIndex = {}
indexToWord = []

ids = []

for word in sentence:
    if word not in wordToIndex:
        indexToWord.append(word)
        idx = len(indexToWord) - 1
        wordToIndex[word] = idx
    else:
        idx = wordToIndex[word]

    if word in embeddings:
        vectors[idx] = embeddings[word]

    ids.append(idx)

After processing your entire corpus the vectors dict contains a idx to vector pair which you can convert to a numpy array (i.e. embedding_weights = np.asarray(list(vectors.values()), dtype=np.float32)) and pass to the embedding layer.
Obviously the example above does not handle unknown tokens or words that don’t have a vector in the embeddings you are using which you should handle
Final note. You could either stop the gradients from flowing on the embedding layer thus keeping your embeddings frozen or not by using the requires_grad variable in the embedding weights