You should look up nn.Embedding. Word embeddings are the de-facto method to vectorize words for most network models and particularly RNNs.
An embedding is a matrix of shape (vocab_size, embed_dim). They also make the use easier since they only use word indexes as input instead of one-hot vectors. Note that multiplying a matrix with a one-hot vector simply selects the corresponding row/column anyway.